Cluster-autoscaler seems to launch randomly a node when PVC is already created.

[payxn]

Which component are you using?: Cluster autoscaler

What version of the component are you using?: Chart version: 9.37.0

Bug description : I'm testing a scale-up of a stateful set where the PVC was already created. It takes some time (more time than a normal scale up) to scale up a node in the same availability zone as the PVC. Is this normal? Additionally, sometimes it scales a node that's not in the same availability zone and then rescales a node after a while (seemingly randomly). Is this also normal?

What did you expect to happen?: Not taking that much time to launch a node with a PVC already created. Also, not launching a node in a different AZ than the POD & PVC.

How to reproduce it (as minimally and precisely as possible): Just launch a statefulset with a PVC. Then, downscale the sts so cluster-autoscaler will remove the node (because of no activity), then rescale the statefulset.

Anything else we need to know?: My cluster-autoscaler configuration

cluster-autoscaler:
  autoDiscovery:
    clusterName: eks-cluster
  awsRegion: us-east-1
  cloudProvider: aws
  extraArgs:
    balance-similar-node-groups: true
    balancing-ignore-label_1: alpha.eksctl.io/nodegroup-name
    balancing-ignore-label_2: alpha.eksctl.io/instance-id
    balancing-ignore-label_3: alpha.eksctl.io/cluster-name
    expander: least-waste
    scale-down-delay-after-add: 1m
    scale-down-unneeded-time: 1m
    skip-nodes-with-system-pods: false
    scan-interval: 10s
  rbac:
    create: true
  serviceMonitor:
    enabled: false
  image:
    pullPolicy: Always
    tag: v1.28.6

My node group :

managedNodeGroups:
  # Private generic worker std xlarge
  - name: private-std-xlarge-1-v1-26        
    minSize: 1
    maxSize: 10
    volumeSize: 50
    volumeType: gp3  
    spot: false
    updateConfig:
      maxUnavailable: 1
    instanceTypes: ["m5a.large"]
    availabilityZones: [us-east-1a, us-east-1b, us-east-1c]
    desiredCapacity: 1    
    iam:
      withAddonPolicies:
        autoScaler: true
        externalDNS: false
        ebs: true
        efs: true
        cloudWatch: true    

[adrianmoisey]

/area cluster-autoscaler

[fgksgf]

Your issue seems be similar to this.

It's recommended to use separate node groups per zone.

[payxn]

Actually I don't have any issue (so I don't think that is similar), it works, but weirdly or slowly. Having one one group per AZ works the same.