I've had a pod stay in a pending state for a long time today, even thought it had plenty of resources to take advantage of in one of my cluster's nodes. When I tried to debug the issue I found out that the node is in an unschedulable state, and when I looked at the logs I saw no new nodes are being added with the autoscaler showing this log message -
2018-04-22 13:28:26,960 - autoscaler.cluster - INFO - KubePod(some-namespace, some-api-588f77c84d-6p6rj) fits on node-name-111111-3
The autoscaler should ignore unschedulable nodes when calculating the cluster's capacity IMO.
Also - I think the node was put into an unschedulable state by the autoscaler, yet wasn't deleted for some reason. but I'm not sure about that yet. Will update if I find more evidence for that behaviour.
MirzaSikander
commented
6 years ago
I've had a pod stay in a pending state for a long time today, even thought it had plenty of resources to take advantage of in one of my cluster's nodes. When I tried to debug the issue I found out that the node is in an
unschedulablestate, and when I looked at the logs I saw no new nodes are being added with the autoscaler showing this log message -The autoscaler should ignore unschedulable nodes when calculating the cluster's capacity IMO. Also - I think the node was put into an unschedulable state by the autoscaler, yet wasn't deleted for some reason. but I'm not sure about that yet. Will update if I find more evidence for that behaviour.