pulumi / pulumi-azure-native

Azure Native Provider
Apache License 2.0
126 stars 33 forks source link

AgentPool resource demands setting of Count property when updating an autoscaled pool from non-autoscaled #2943

Open worldspawn opened 9 months ago

worldspawn commented 9 months ago

What happened?

Trying to create an AgentPool where EnableAutoScaling is true. MaxCount, MinCount are set. Pulumi fails with

error: Code="InvalidParameter" Message="Required parameter properties.count is missing (null)." Target="properties.count"

Example

var tcServerPool = new AgentPool("teamcity-server", new AgentPoolArgs()
        {
            AgentPoolName = "tcserver",
            VmSize = "Standard_DS4_v2",
            EnableAutoScaling = true,
            MinCount = 0,
            MaxCount = 1,
            OsType = OSType.Linux,
            OsDiskSizeGB = 100,
            VnetSubnetID = subnetId,
            ResourceGroupName = groupName,
            ResourceName = clusterName,
            NodeTaints = new List<string>
            {
                "teamcity/server:NoSchedule"
            }
        });

Output of pulumi about

running 'dotnet build -nologo .'
  Determining projects to restore...

  All projects are up-to-date for restore.

  teamcity -> C:\Users\critc\source\drawboard-shared-infra\pulumi\teamcity\bin\Debug\net6.0\teamcity.dll

Build succeeded.

    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.96

'dotnet build -nologo .' completed successfully
CLI
Version      3.94.2
Go Version   go1.21.3
Go Compiler  gc

Plugins
NAME          VERSION
azure-native  2.21.0
dotnet        unknown
kubernetes    4.5.5
random        4.14.0

Host
OS       Microsoft Windows 11 Pro
Version  10.0.22621 Build 22621
Arch     x86_64

This project is written in dotnet: executable='C:\Program Files\dotnet\dotnet.exe' version='8.0.100'

Current Stack: drawboard/teamcity/dev

TYPE                                                 URN
pulumi:pulumi:Stack                                  urn:pulumi:dev::teamcity::pulumi:pulumi:Stack::teamcity-dev
pulumi:providers:pulumi                              urn:pulumi:dev::teamcity::pulumi:providers:pulumi::default
pulumi:providers:random                              urn:pulumi:dev::teamcity::pulumi:providers:random::default_4_14_0
random:index/randomPassword:RandomPassword           urn:pulumi:dev::teamcity::random:index/randomPassword:RandomPassword::sqlpassword
pulumi:pulumi:StackReference                         urn:pulumi:dev::teamcity::pulumi:pulumi:StackReference::shared-infra
pulumi:providers:kubernetes                          urn:pulumi:dev::teamcity::pulumi:providers:kubernetes::k8s
kubernetes:core/v1:Namespace                         urn:pulumi:dev::teamcity::kubernetes:core/v1:Namespace::teamcity
pulumi:providers:azure-native                        urn:pulumi:dev::teamcity::pulumi:providers:azure-native::default_2_21_0
kubernetes:core/v1:Secret                            urn:pulumi:dev::teamcity::kubernetes:core/v1:Secret::teamcitysa
kubernetes:core/v1:ConfigMap                         urn:pulumi:dev::teamcity::kubernetes:core/v1:ConfigMap::tc-config
kubernetes:rbac.authorization.k8s.io/v1:Role         urn:pulumi:dev::teamcity::kubernetes:rbac.authorization.k8s.io/v1:Role::teamcity-manageagents
kubernetes:apps/v1:StatefulSet                       urn:pulumi:dev::teamcity::kubernetes:apps/v1:StatefulSet::teamcity
azure-native:network:RecordSet                       urn:pulumi:dev::teamcity::azure-native:network:RecordSet::teamcity
kubernetes:core/v1:ServiceAccount                    urn:pulumi:dev::teamcity::kubernetes:core/v1:ServiceAccount::teamcity
kubernetes:rbac.authorization.k8s.io/v1:RoleBinding  urn:pulumi:dev::teamcity::kubernetes:rbac.authorization.k8s.io/v1:RoleBinding::teamcity-manageagents
azure-native:containerservice:AgentPool              urn:pulumi:dev::teamcity::azure-native:containerservice:AgentPool::teamcity-agents
kubernetes:core/v1:Service                           urn:pulumi:dev::teamcity::kubernetes:core/v1:Service::teamcity
kubernetes:networking.k8s.io/v1:Ingress              urn:pulumi:dev::teamcity::kubernetes:networking.k8s.io/v1:Ingress::teamcity
azure-native:containerservice:AgentPool              urn:pulumi:dev::teamcity::azure-native:containerservice:AgentPool::teamcity-server

Found no pending operations associated with drawboard/dev

Backend
Name           pulumi.com
URL            https://app.pulumi.com/samcritchley
User           samcritchley
Organizations  samcritchley, drawboard
Token type     personal

Dependencies:
NAME                VERSION
Pulumi              3.59.0
Pulumi.AzureNative  2.21.0
Pulumi.Kubernetes   4.5.5
Pulumi.Random       4.14.0

Pulumi locates its logs in C:\Users\critc\AppData\Local\Temp by default

Additional context

Specifying all 3 values is accepted on create:

            Count = 0,
            MinCount = 0,
            MaxCount = 1,
            EnableAutoScaling = true

Then set EnableAutoScaling to false

Fails with

 error: Code="InvalidParameter" Message="AgentPool 'tcagents' has 'agentPoolProfile.minCount' set which requires that 'agentPoolProfile.enableAutoscaling' is true and it is false"

So if we remove MinCount and MaxCount

EnableAutoScaling = false,
Count = 0,

Succeeds. Now a not-auto-scaled pool with 0 instances. Lets go back to auto scaling:

            EnableAutoScaling = true,
            Count = 0,
            MinCount = 0,
            MaxCount = 4,

Fails with

error: Code="InvalidParameter" Message="AgentPool 'tcagents' has set auto scaling as enabled but is not on Virtual Machine Scale Sets, this is not allowed. Please see https://aka.ms/aks-vmss-enablement for more details."

Hmm... lets set that then

Type = AgentPoolType.VirtualMachineScaleSets,

And it works. I didn't need to set this when I created the resource as an auto-scaling agent pool and need to when changing from non auto to auto. I get the sense the provider has done "something" for me when I created the resource and that same something is not happening in this update scenario.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

mjeffryes commented 9 months ago

Looks like AgentPool is actually an azure-native resource; I think this behavior is all specific to azure so moving the issue there.

To clarify, it looks like you're able to create the autoscaling AgentPool by just setting EnableAutoScaling, but when you update an AgentPool that was created with EnableAutoScaling = false, you have to set both EnableAutoScaling and Type = AgentPoolType.VirtualMachineScaleSets for it to update correct?