Kubernetes Cluster Setup

Before deploying Akri, you must have a Kubernetes cluster (v1.16 or higher) running with kubectl and Helm installed. Akri is Kubernetes native, so it should run on most Kubernetes distributions. This document provides cluster setup instructions for the three Kubernetes distributions that all of our end-to-end tests run on.

Note: All nodes must be Linux on amd64, arm64v8, or arm32v7.

Install Kubernetes Distribution

Kubernetes

1. Reference Kubernetes documentation for instructions on how to install Kubernetes. See Akri's release notes to see what versions of Kubernetes Akri has been tested on.

2. Install Helm for deploying Akri.

    sudo apt install -y curl
    curl -L https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

Note: To enable workloads on a single-node cluster, remove the master taint.

kubectl taint nodes --all node-role.kubernetes.io/master-

K3s

1. Install K3s. The following will install the latest K3s version. Reference Akri's release notes to see what versions of K3s Akri has been tested on.

      curl -sfL https://get.k3s.io | sh -

   Note: Optionally specify a version with the INSTALL_K3S_VERSION env var as follows: curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.21.5+k3s1 sh -

2. Grant admin privilege to access kube config.

    sudo addgroup k3s-admin
    sudo adduser $USER k3s-admin
    sudo usermod -a -G k3s-admin $USER
    sudo chgrp k3s-admin /etc/rancher/k3s/k3s.yaml
    sudo chmod g+r /etc/rancher/k3s/k3s.yaml
    su - $USER

3. Check K3s status.

    kubectl get node

4. Install Helm.

    export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
    sudo apt install -y curl
    curl -L https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

5. If desired, add nodes to your cluster by running the K3s installation script with the K3S_URL and K3S_TOKEN environment variables. See K3s installation documentation for more details.

MicroK8s

1. Install MicroK8s. The following will install the latest MicroK8s version. Add --channel=$VERSION/stable to specify as specific Kubernetes version. Reference Akri's release notes to see what versions of MicroK8s Akri has been tested on.

    snap install microk8s --classic

2. Grant admin privilege for running MicroK8s commands.

    sudo usermod -a -G microk8s $USER
    sudo chown -f -R $USER ~/.kube
    su - $USER

3. Check MicroK8s status.

    microk8s status --wait-ready

4. Enable CoreDNS, Helm and RBAC for MicroK8s.

    microk8s enable dns helm3 rbac

5. If you don't have an existing kubectl and helm installations, add aliases. If you do not want to set an alias, add microk8s in front of all kubectl and helm commands.

    alias kubectl='microk8s kubectl'
    alias helm='microk8s helm3'

6. By default, MicroK8s does not allow Pods to run in a privileged context. None of Akri's components run privileged; however, if your custom broker Pods do in order to access devices for example, enable privileged Pods like so:

    echo "--allow-privileged=true" >> /var/snap/microk8s/current/args/kube-apiserver
    microk8s.stop
    microk8s.start

7. If desired, reference MicroK8's documentation to add additional nodes to the cluster.

Configure crictl

Akri depends on crictl to track some Pod information. In order to use it, the Agent must know where the container runtime socket lives. This can be configured with Akri's Helm chart either directly by setting agent.host.containerRuntimeSocket or indirectly by specifying the Kubernetes distribution that is being used (kubernetesDistro=k3s|microk8s|k8s). If a distribution is specified, then the appropriate default will be used. If no distribution or runtime is specified, the k8s default is used.

Akri recommends setting this choice as an AKRI_HELM_CRICTL_CONFIGURATION environment variable and then adding the variable to each Akri installation like so:

  helm install akri akri-helm-charts/akri \
     $AKRI_HELM_CRICTL_CONFIGURATION

The following are the recommended settings based on Kubernetes distribution.

Kubernetes

To use the default standard Kubernetes container runtime socket /run/containerd/containerd.sock, set k8s as the distribution.

export AKRI_HELM_CRICTL_CONFIGURATION="--set kubernetesDistro=k8s"

K3s/RKE2

To use the default K3s container runtime socket /run/k3s/containerd/containerd.sock, set k3s as the distribution.

export AKRI_HELM_CRICTL_CONFIGURATION="--set kubernetesDistro=k3s"

MicroK8s

To use the default MicroK8s container runtime socket /var/snap/microk8s/common/run/containerd.sock, set microk8s as the distribution.

export AKRI_HELM_CRICTL_CONFIGURATION="--set kubernetesDistro=microk8s"

Other

A specific container runtime socket can be set in the agent.host.containerRuntimeSocket value.

export AKRI_HELM_CRICTL_CONFIGURATION="--set agent.host.containerRuntimeSocket=/container/runtime.sock"