Issues when installing and how I solved them

[PieterScheffers]

First I must say thank you for this great guide! It gives a very good explanation how to install everything and how it fits together. When installing this I came across some issues. So here I want to contribute back and share the fixes I found that made it work for me.

1. After a reboot wireguard would not be started. (wg show would give an empty result). This can be solved by reinstalling the linux-headers (apt-get install -y linux-headers-$(uname -r)) and rebooting. I had this issue coming back multiple times, so I put the install in a oneshot systemd service and put After=install-linux-headers in the wg-quick@.service files.

2. After kubeadm init I would initialize the weave network. But when joining the slaves with kubeadm join, the slave nodes would stay in the 'not ready' state. When investigating I found it couldn't start the pod network. This could be solved by adding to the masterconfiguration for kubeadm:

   apiVersion: kubeadm.k8s.io/v1beta1
   kind: InitConfiguration
   networking:
   podSubnet: 10.32.0.0/12

   and the CIDR you want to use can also be passed to weave with:

   kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')&env.IPALLOC_RANGE=10.32.0.0%2F12"

3. There would already be a route for the 10.32.0.0/12 (weave) network. So I couldn't setup the extra ip route. (ip route)

4. As last step I installed the UFW firewall. The cluster would work without it, but after activating the firewall it wouldn't work anymore. The error was because kubelet was being contacted via the Scaleway private IP. To make kubelet be connected to via the Wireguard network I had to add --node-ip=10.0.1.1 to the /var/lib/kubelet/kubeadm-flags.env file. (The kubelet systemd file reads the kubelet arguments from that file)

[cbioley]

@PieterScheffers Thank you Pieter!

I was having an issue with a node marked as NotReady after an upgrade.

I reinstalled the headers and now everything is back to normal 🥳

[pstadler]

@PieterScheffers did you use the provisioning repository or did you encounter these problems during a manual setup?

[PieterScheffers]

@pstadler I did a manual setup, so it could very well be that I did something wrong.

[kotluk]

I want to mention that networking option doesn't work for me under InitConfiguration I put this option under ClusterConfiguration and it worked

apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
networking:
    podSubnet: 10.32.0.0/12

Thank you, Pieter!