Feature request: Support for setting multiple expanders for AKS autoscaler

yaraskm commented 2 years ago

The Cluster Autoscaler allows multiple expander profiles to be used at once, so that if the first expander matches more than one Node Pool, the next expander is used to narrow down the selection.

Currently however, the REST API requires that only one expander can be set:

"expander": {
              "type": "string",
              "enum": [
                "least-waste",
                "most-pods",
                "priority",
                "random"
              ],
              "x-ms-enum": {
                "name": "expander",
                "modelAsString": true,
                "values": [
                  {
                    "value": "least-waste",
                    "description": "Selects the node group that will have the least idle CPU (if tied, unused memory) after scale-up. This is useful when you have different classes of nodes, for example, high CPU or high memory nodes, and only want to expand those when there are pending pods that need a lot of those resources."
                  },
                  {
                    "value": "most-pods",
                    "description": "Selects the node group that would be able to schedule the most pods when scaling up. This is useful when you are using nodeSelector to make sure certain pods land on certain nodes. Note that this won't cause the autoscaler to select bigger nodes vs. smaller, as it can add multiple smaller nodes at once."
                  },
                  {
                    "value": "priority",
                    "description": "Selects the node group that has the highest priority assigned by the user. It's configuration is described in more details [here](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/expander/priority/readme.md)."
                  },
                  {
                    "value": "random",
                    "description": "Used when you don't have a particular need for the node groups to scale differently."
                  }
                ]
              },
              "title": "The expander to use when scaling up",
              "description": "If not specified, the default is 'random'. See [expanders](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-are-expanders) for more information."
            },

This API should be expanded to allow a list of profiles to be active at once.

This came up through a feature request on the azurerm Terraform provider