search

ITRS-Group / check_k8s

Check plugin for kubernetes
GNU General Public License v2.0
7 stars 6 forks source link

k8s nodes status shows as degraded #68

Open CamZie opened 6 months ago

CamZie commented 6 months ago

Hello,

We recently upgraded our rancher and kubernetes version:

After the upgrade this k8s checks detects all our nodes as "degraded". However, upon analysing we could not find any issues with any of the nodes since the status and availability from rancher are all fine.

This is the output we receive when running the check on the nodes:

/opt/check_k8s/check_k8s.py --port 443 --host <HOSTNAME> --resource nodes --insecure --token <TOKEN> --debug

[DEBUG] Node node1: condition EtcdIsVoter since 2024-04-20T04:25:35Z
[DEBUG] Node node2: condition EtcdIsVoter since 2024-05-08T04:59:03Z
[DEBUG] Node node3: condition EtcdIsVoter since 2024-05-08T04:46:04Z
WARNING - One or more warnings encountered
Node node1: condition EtcdIsVoter since 2024-04-20T04:25:35Z
Node node2: condition EtcdIsVoter since 2024-05-08T04:59:03Z
Node node3: condition EtcdIsVoter since 2024-05-08T04:46:04Z|available=0 unavailable=0 degraded=3 unschedulable=0

Running a pods check with the same commands results to all successful:

/opt/check_k8s/check_k8s.py --port 443 --host <HOSTNAME> --resource pods --insecure --token <TOKEN> --debug

...
OK - All checks were successful
...
|available=49 unavailable=0 degraded=0 pending=0

The status of the nodes are all fine:

NAME    STATUS   ROLES                       AGE    VERSION
node1   Ready    control-plane,etcd,master   427d   v1.27.11+rke2r1
node2   Ready    control-plane,etcd,master   427d   v1.27.11+rke2r1
node3   Ready    control-plane,etcd,master   427d   v1.27.11+rke2r1

Any help would be very much appreciated.

MMS-RBKN commented 2 months ago

Maybe https://github.com/ITRS-Group/check_k8s/compare/ITRS-Group:master...G-Widianto:master?expand=1 helps you, too.