Allow updating node pools using ARM template

[aelij]

What happened: Currently the only option to add a node pool in an ARM template is by creating a separate child resource (.../providers/Microsoft.ContainerService/managedClusters/aks1/agentPools/p2). This presents a problem when trying to apply an update to the primary (system) node pool which requires recreating it, for example, to allow it to join an existing subnet or enable "encryption at host".

If we were to add the new agent pool using a child resource in the template, the template will no longer be idempotent (i.e. it won't be able to deploy a new clean environment) and also the template would not clean up the old pool. It forces us to use scripts to complement ARM.

ARM deployments were made to be idempotent and this essentially breaks it.

Update: Another non-idempotent related issue:

Code: OperationNotAllowed Message: Updating Kubernetes version and agent node scaling are mutually exclusive operations.

AKS should be able to handle these kind of updates on its own.

What you expected to happen: Allow updating node pools using the agentPoolProfiles array of the managedClusters type.

Environment:

• Kubernetes version (use kubectl version): v1.18.14
• Size of cluster (how many worker nodes are in the cluster?) 5
• General description of workloads in the cluster: HTTP microservices

[fschmied]

Not stale.

[microsoft-github-policy-service[bot]]

This issue has been automatically marked as stale because it has not had any activity for 21 days. It will be closed if no further activity occurs within 7 days of this comment.

[fschmied]

Not stale