Can't start kubelet v1.24.3

[jeffreymlewis]

I've been using this wonderful guide to install kubernetes v1.24.3 in vagrant. I've gotten over most of the hurdles but I cannot get kubelet running on the worker nodes. The error seems to be this. Let me know if you have any ideas.

Aug 07 19:14:53 worker-1 kubelet[5585]: W0807 19:14:53.670691 5585 clientconn.go:1331] [core] grpc: addrConn.createTransport failed to connect to { \<nil> 0 \<nil>}. Err: connection error: desc = "transport: Error while dialing dial unix: missing address". Reconnecting...

Aug 07 19:14:53 worker-1 kubelet[5585]: Error: failed to run Kubelet: unable to determine runtime API version: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial unix: missing address"

I'm using the following config and systemd unit files

(/var/lib/kubelet/kubelet-config.yaml)

kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
  x509:
    clientCAFile: "/var/lib/kubernetes/ca.crt"
authorization:
  mode: Webhook
clusterDomain: "cluster.local"
clusterDNS:
  - "10.96.0.10"
resolvConf: "/run/systemd/resolve/resolv.conf"
runtimeRequestTimeout: "15m"
tlsCertFile: "/var/lib/kubelet/worker-1.crt"
tlsPrivateKeyFile: "/var/lib/kubelet/worker-1.key"

(/etc/systemd/system/kubelet.service)

[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/local/bin/kubelet \
  --config=/var/lib/kubelet/kubelet-config.yaml \
  --kubeconfig=/var/lib/kubelet/kubeconfig \
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

[jeffreymlewis]

The issue is that Kubernetes 1.24+ no longer supports Dockershim, so we need to install cri-docker and configure kubelet to use it. This is a simple (though tedious) process.

https://github.com/Mirantis/cri-dockerd

Another hint. You need to build cri-dockerd on one of the master vagrant nodes, then scp the binary to the worker nodes. The worker nodes themselves don't have sufficient ram for the go build... step.

Once you have cri-dockerd installed and running on the workers, add the --container-runtime-endpoint flag in /etc/systemd/system/kubelet.service and you're good to go!

[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/local/bin/kubelet \
  --config=/var/lib/kubelet/kubelet-config.yaml \
  --kubeconfig=/var/lib/kubelet/kubeconfig \
  --container-runtime-endpoint=/run/cri-dockerd.sock \
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

[fireflycons]

Hi @jeffreymlewis

We are in the process of performing a full refresh of this to v1.24. There are a lot of changes involved! This is currently in review and should go live soon. We also intend to update this repo to keep pace with the version currently used by CKA exam, so will advance to 1.25 soon.

Watch this space.