Failed to create Kubernetes cluster, problem with kubelet service

[graziadonghia]

Hello, I'm trying to create a confidential computing Kubernetes cluster by following your guide. My operating system is Ubuntu 20.04 and I have successfully installed SGX driver, SGX SDK and SGX PSW, together with all the required packages to use Inclavare containers. However whenever I try to create the cluster with kubeadm init this is the output:

Unfortunately, an error has occurred:
    timed out waiting for the condition

This error is likely caused by:
    - The kubelet is not running
    - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
    - 'systemctl status kubelet'
    - 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
    - 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
    Once you have found the failing container, you can inspect its logs with:
    - 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher

The problem is related to the kubelet service, which sometimes works and sometimes not (I don't know why), giving this output:

● kubelet.service - kubelet: The Kubernetes Node Agent
     Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/kubelet.service.d
             └─10-kubeadm.conf
     Active: activating (auto-restart) (Result: exit-code) since Mon 2022-08-29 17:57:11 CEST; 2s ago
       Docs: https://kubernetes.io/docs/home/
    Process: 3930445 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_EXT>
   Main PID: 3930445 (code=exited, status=1/FAILURE)

[hustliyilin]

Hi @graziadonghia ,

Thanks for your question. Maybe you can follow this CI/CD guide to run rune and shim-rune with k8s in ubuntu

[graziadonghia]

It still doesn't work. I think it's because of the content of 10-kubeadm.conf file, because the kubelet service runs with SUCCESS code if I insert only $KUBELET_EXTRA_ARGS (which are --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock because otherwise it doesn't even start). With this configuration for ExecStart in 10-kubeadm.conf:

ExecStart = /usr/bin/kubelet  --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock

the kubelet service works but I cannot create the cluster. the error message is: please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/cni/. How can I deal with cni? Again, I followed multiple times the guide together with the github actions, I don't know what to do.

Is it possible to create the cluster using other tools? like K3s or minikube?

[hustliyilin]

You can use cri-tools to create the cluster.