search

canonical / ua-reviewkit

3 stars 6 forks source link

sonobuoy default run fails with Unschedulable nodes #27

Open lolwww opened 1 year ago

lolwww commented 1 year ago

With Kubernetes 1.24 deployed with Juju/MAAS on baremetal default way of launching sonobuoy like this:

./sonobuoy run

Fails with the following result:

$ ./sonobuoy results $results Plugin: e2e Status: failed Total: 1 Passed: 0 Failed: 1 Skipped: 0

from the e2e log: Jan 12 14:50:28.313: INFO: ==== node wait: 3 out of 6 nodes are ready, max notReady allowed 0. Need 3 more before starting. Jan 12 14:50:58.311: INFO: Unschedulable nodes= 3, maximum value for starting tests= 0 Jan 12 14:50:58.311: INFO: -> Node k8s-control-plane-1 [[[ Ready=true, Network(available)=false, Taints=[{juju.is/kubernetes-control-plane true NoSchedule }], NonblockingTaints=node-role.kubernetes.io/control-plane,node-role.kubernetes.io/master ]]]

The workaround is to include the suggested taint to non-blocking-taints arg like this:

./sonobuoy run --plugin-env=e2e.E2E_EXTRA_ARGS=--non-blocking-taints=juju.is/kubernetes-control-plane,true,NoSchedule

So this might be a good idea to launch it the same way in kubernetes-extra-checks.sh

VariableDeclared commented 1 year ago

I suspect this is from the recent changes which make control-plane nodes scheduable. There's a taint on the nodes which prevents scheduling, which can be removed, or sonobouy tests can be tuned to ignore this - I think.