kubernetes / autoscaler

Autoscaling components for Kubernetes
Apache License 2.0
7.94k stars 3.93k forks source link

Scaling up EKS Managed Node Group using Windows nodes #7082

Open ojundt opened 1 month ago

ojundt commented 1 month ago

Which component are you using?:

cluster-autoscaler

What version of the component are you using?: 1.29.0

What k8s version are you using (kubectl version)?: 1.29

What environment is this in?: AWS EKS

What did you expect to happen?: When I schedule a pod with nodeSelector=kubernetes.io/os: windows on a Windows-based EKS Managed Node Group that's currently running at 0 nodes, I expect cluster autoscaler to scale up the node group from zero.

What happened instead?: Cluster autoscaler didn't scale up the node group from zero. Logs indicate that the nodegroup doesn't match the nodeselector.

My hunch: Cluster Autoscaler doesn't know that EKS Managed Node Group will inject the kubernetes.io/os=windows node label because it's not part of the DescribeNodeGroup response that's being used to feed the cache.

How to reproduce it (as minimally and precisely as possible):

  1. Create a Managed Node Group in EKS using a Windows AMI type (e.g. WINDOWS_CORE_2022_x86_64) with a min count of 0.
  2. Make sure no nodes are running in that group yet.
  3. Schedule a pod with:
    nodeSelector:
        kubernetes.io/os: windows
  4. See cluster autoscaler not starting a new node

Anything else we need to know?:

adrianmoisey commented 1 month ago

/area cluster-autoscaler