canonical / microk8s

MicroK8s is a small, fast, single-package Kubernetes for datacenters and the edge.
https://microk8s.io
Apache License 2.0
8.52k stars 773 forks source link

Service snap.microk8s.daemon-kubelite is not running on woker node, resulting in a non-ready node status #4685

Open Darth-Weider opened 1 month ago

Darth-Weider commented 1 month ago

After adding 3 nodes (1 HA node, 2 worker nodes) to an existing HA cluster, the cluster initially functioned as expected. However, following a reboot of all nodes after patching (from v1.29.8 -> 1.29.9), the worker nodes became non-ready. Investigation revealed that the snap.microk8s.daemon-kubelite service was not running on these nodes. Removed 2 worker nodes and add them back as HA nodes, everything just works

louiseschmidtgen commented 1 month ago

Hello @Darth-Weider,

Thank you for raising your issue.

Could you please share the inspection report from your nodes? Is this issue reproducible with a snap refresh from v1.29.8 to v1.29.9?

With kind regards, Louise

Darth-Weider commented 1 month ago

Hi @louiseschmidtgen

Unfortunately the inspection report was no longer available (I did a snap remove microk8s before add them back as HA node). I took some screen shots

Screenshot 2024-09-25 at 9 34 31 PM Screenshot 2024-09-27 at 2 23 05 PM Screenshot 2024-09-25 at 9 17 17 PM

Not sure if the discrepancy between the older Kubernetes version (v1.29.8) on the datastore master nodes and the newer version (v1.29.9) on the new nodes could be contributing factor to my issue. I have updated my SOP to include patching datastore master nodes to the latest k8s release before adding new nodes to the cluster.