az aks update --enable-keda fails

m42jbz commented 2 years ago

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name

az aks update --enable-keda --cluster-name $clusterName --resource-group $resourceGroup

Errors:

Argument '--enable-keda' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The behavior of this command has been altered by the following extension: aks-preview
The command failed with an unexpected error. Here is the traceback:
'AKSPreviewManagedClusterUpdateDecorator' object has no attribute 'get_yes'
Traceback (most recent call last):
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 663, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 697, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 333, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "C:\Users\[REDACTED]\.azure\cliextensions\aks-preview\azext_aks_preview\custom.py", line 831, in aks_update
    mc = aks_update_decorator.update_mc_profile_preview()
  File "C:\Users\[REDACTED]\.azure\cliextensions\aks-preview\azext_aks_preview\managed_cluster_decorator.py", line 3160, in update_mc_profile_preview
    mc = self.update_outbound_type_in_network_profile(mc)
  File "C:\Users\[REDACTED]\.azure\cliextensions\aks-preview\azext_aks_preview\managed_cluster_decorator.py", line 2663, in update_outbound_type_in_network_profile
    if multizoned and not self.get_yes() and not prompt_y_n("\n" + CONST_OUTBOUND_MIGRATION_MULTIZONE_TO_NATGATEWAY_MSG, default="y"):
AttributeError: 'AKSPreviewManagedClusterUpdateDecorator' object has no attribute 'get_yes'
To open an issue, please run: 'az feedback'

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Have a an AKS cluster deployed before enabling the feature and updating the resource provider (deployed via pulumi, ARM API version V20220301)
Follow the instructions in the docs: https://learn.microsoft.com/en-us/azure/aks/keda-deploy-add-on-cli

Expected Behavior

KEDA is enabled.

Environment Summary

Windows-10-10.0.19044-SP0
Python 3.10.5
Installer: MSI

azure-cli 2.41.0

Extensions:
account 0.2.3
aks-preview 0.5.112
automation 0.1.1
cli-translator 0.3.0
desktopvirtualization 0.2.0
dev-spaces 1.0.6
resource-graph 2.1.0
storage-preview 0.8.3

Dependencies:
msal 1.20.0b1
azure-mgmt-resource 21.1.0b1

Additional Context

Output of az aks show:

{
  "aadProfile": {
    "adminGroupObjectIDs": [
      "[REDACTED]"
    ],
    "adminUsers": null,
    "clientAppId": null,
    "enableAzureRbac": null,
    "managed": true,
    "serverAppId": null,
    "serverAppSecret": null,
    "tenantId": "[REDACTED]"
  },
  "addonProfiles": {
    "azurepolicy": {
      "config": null,
      "enabled": false,
      "identity": null
    },
    "httpApplicationRouting": {
      "config": null,
      "enabled": false,
      "identity": null
    },
    "omsagent": {
      "config": null,
      "enabled": false,
      "identity": null
    }
  },
  "agentPoolProfiles": [
    {
      "availabilityZones": [
        "1",
        "2",
        "3"
      ],
      "capacityReservationGroupId": null,
      "count": 1,
      "creationData": null,
      "currentOrchestratorVersion": "1.23.12",
      "enableAutoScaling": true,
      "enableCustomCaTrust": false,
      "enableEncryptionAtHost": false,
      "enableFips": false,
      "enableNodePublicIp": false,
      "enableUltraSsd": false,
      "gpuInstanceProfile": null,
      "hostGroupId": null,
      "kubeletConfig": null,
      "kubeletDiskType": "OS",
      "linuxOsConfig": null,
      "maxCount": 5,
      "maxPods": 60,
      "messageOfTheDay": null,
      "minCount": 0,
      "mode": "User",
      "name": "[REDACTED]",
      "networkProfile": null,
      "nodeImageVersion": "AKSWindows-2019-17763.3534.221019",
      "nodeLabels": null,
      "nodePublicIpPrefixId": null,
      "nodeTaints": null,
      "orchestratorVersion": "1.23.12",
      "osDiskSizeGb": 128,
      "osDiskType": "Managed",
      "osSku": "Windows2019",
      "osType": "Windows",
      "podSubnetId": null,
      "powerState": {
        "code": "Running"
      },
      "provisioningState": "Succeeded",
      "proximityPlacementGroupId": null,
      "scaleDownMode": "Delete",
      "scaleSetEvictionPolicy": null,
      "scaleSetPriority": null,
      "spotMaxPrice": null,
      "tags": null,
      "type": "VirtualMachineScaleSets",
      "upgradeSettings": {
        "maxSurge": null
      },
      "vmSize": "Standard_E4as_v5",
      "vnetSubnetId": "[REDACTED]",
      "windowsProfile": null,
      "workloadRuntime": null
    },
    {
      "availabilityZones": null,
      "capacityReservationGroupId": null,
      "count": 1,
      "creationData": null,
      "currentOrchestratorVersion": "1.23.12",
      "enableAutoScaling": true,
      "enableCustomCaTrust": false,
      "enableEncryptionAtHost": false,
      "enableFips": false,
      "enableNodePublicIp": false,
      "enableUltraSsd": false,
      "gpuInstanceProfile": null,
      "hostGroupId": null,
      "kubeletConfig": null,
      "kubeletDiskType": "OS",
      "linuxOsConfig": null,
      "maxCount": 5,
      "maxPods": 80,
      "messageOfTheDay": null,
      "minCount": 0,
      "mode": "User",
      "name": "[REDACTED]",
      "networkProfile": null,
      "nodeImageVersion": "AKSWindows-2019-containerd-17763.3534.221019",
      "nodeLabels": null,
      "nodePublicIpPrefixId": null,
      "nodeTaints": null,
      "orchestratorVersion": "1.23.12",
      "osDiskSizeGb": 128,
      "osDiskType": "Managed",
      "osSku": "Windows2019",
      "osType": "Windows",
      "podSubnetId": null,
      "powerState": {
        "code": "Running"
      },
      "provisioningState": "Succeeded",
      "proximityPlacementGroupId": null,
      "scaleDownMode": "Delete",
      "scaleSetEvictionPolicy": null,
      "scaleSetPriority": null,
      "spotMaxPrice": null,
      "tags": null,
      "type": "VirtualMachineScaleSets",
      "upgradeSettings": {
        "maxSurge": null
      },
      "vmSize": "Standard_E4s_v5",
      "vnetSubnetId": "[REDACTED]",
      "windowsProfile": null,
      "workloadRuntime": null
    },
    {
      "availabilityZones": [
        "1",
        "2",
        "3"
      ],
      "capacityReservationGroupId": null,
      "count": 2,
      "creationData": null,
      "currentOrchestratorVersion": "1.23.12",
      "enableAutoScaling": true,
      "enableCustomCaTrust": false,
      "enableEncryptionAtHost": null,
      "enableFips": false,
      "enableNodePublicIp": false,
      "enableUltraSsd": null,
      "gpuInstanceProfile": null,
      "hostGroupId": null,
      "kubeletConfig": null,
      "kubeletDiskType": "OS",
      "linuxOsConfig": null,
      "maxCount": 7,
      "maxPods": 30,
      "messageOfTheDay": null,
      "minCount": 1,
      "mode": "System",
      "name": "[REDACTED]",
      "networkProfile": null,
      "nodeImageVersion": "AKSUbuntu-1804gen2containerd-2022.10.03",
      "nodeLabels": null,
      "nodePublicIpPrefixId": null,
      "nodeTaints": null,
      "orchestratorVersion": "1.23.12",
      "osDiskSizeGb": 128,
      "osDiskType": "Managed",
      "osSku": "Ubuntu",
      "osType": "Linux",
      "podSubnetId": null,
      "powerState": {
        "code": "Running"
      },
      "provisioningState": "Succeeded",
      "proximityPlacementGroupId": null,
      "scaleDownMode": "Delete",
      "scaleSetEvictionPolicy": null,
      "scaleSetPriority": null,
      "spotMaxPrice": null,
      "tags": null,
      "type": "VirtualMachineScaleSets",
      "upgradeSettings": {
        "maxSurge": null
      },
      "vmSize": "Standard_DS2_v2",
      "vnetSubnetId": "[REDACTED]",
      "windowsProfile": null,
      "workloadRuntime": null
    }
  ],
  "apiServerAccessProfile": null,
  "autoScalerProfile": {
    "balanceSimilarNodeGroups": "false",
    "expander": "random",
    "maxEmptyBulkDelete": "10",
    "maxGracefulTerminationSec": "600",
    "maxNodeProvisionTime": "15m",
    "maxTotalUnreadyPercentage": "45",
    "newPodScaleUpDelay": "0s",
    "okTotalUnreadyCount": "3",
    "scaleDownDelayAfterAdd": "10m",
    "scaleDownDelayAfterDelete": "10s",
    "scaleDownDelayAfterFailure": "3m",
    "scaleDownUnneededTime": "60m",
    "scaleDownUnreadyTime": "20m",
    "scaleDownUtilizationThreshold": "0.4",
    "scanInterval": "10s",
    "skipNodesWithLocalStorage": "false",
    "skipNodesWithSystemPods": "true"
  },
  "autoUpgradeProfile": null,
  "azureMonitorProfile": null,
  "azurePortalFqdn": "[REDACTED]",
  "creationData": null,
  "currentKubernetesVersion": "1.23.12",
  "disableLocalAccounts": null,
  "diskEncryptionSetId": null,
  "dnsPrefix": "[REDACTED]",
  "enableNamespaceResources": null,
  "enablePodSecurityPolicy": null,
  "enableRbac": true,
  "extendedLocation": null,
  "fqdn": "[REDACTED]",
  "fqdnSubdomain": null,
  "guardrailsProfile": {
    "excludedNamespaces": null,
    "level": "Off",
    "systemExcludedNamespaces": null,
    "version": null
  },
  "httpProxyConfig": null,
  "id": "[REDACTED]",
  "identity": {
    "principalId": null,
    "tenantId": null,
    "type": "UserAssigned",
    "userAssignedIdentities": {
      "[REDACTED]": {
        "clientId": "[REDACTED]",
        "principalId": "[REDACTED]"
      }
    }
  },
  "identityProfile": {
    "kubeletidentity": {
      "clientId": "[REDACTED]",
      "objectId": "[REDACTED]",
      "resourceId": "[REDACTED]"
    }
  },
  "ingressProfile": null,
  "kubernetesVersion": "1.23.12",
  "linuxProfile": null,
  "location": "westeurope",
  "maxAgentPools": 100,
  "name": "[REDACTED]",
  "networkProfile": {
    "dnsServiceIp": "10.0.0.10",
    "dockerBridgeCidr": "172.17.0.1/16",
    "ebpfDataplane": null,
    "ipFamilies": [
      "IPv4"
    ],
    "kubeProxyConfig": null,
    "loadBalancerProfile": {
      "allocatedOutboundPorts": 0,
      "backendPoolType": "nodeIPConfiguration",
      "effectiveOutboundIPs": [
        {
          "id": "[REDACTED]",
          "resourceGroup": "[REDACTED]"
        }
      ],
      "enableMultipleStandardLoadBalancers": null,
      "idleTimeoutInMinutes": 30,
      "managedOutboundIPs": {
        "count": 1,
        "countIpv6": null
      },
      "outboundIPs": null,
      "outboundIpPrefixes": null
    },
    "loadBalancerSku": "Standard",
    "natGatewayProfile": null,
    "networkMode": null,
    "networkPlugin": "azure",
    "networkPluginMode": null,
    "networkPolicy": null,
    "outboundType": "loadBalancer",
    "podCidr": null,
    "podCidrs": null,
    "serviceCidr": "10.0.0.0/16",
    "serviceCidrs": [
      "10.0.0.0/16"
    ]
  },
  "nodeResourceGroup": "[REDACTED]",
  "oidcIssuerProfile": {
    "enabled": false,
    "issuerUrl": null
  },
  "podIdentityProfile": {
    "allowNetworkPluginKubenet": null,
    "enabled": true,
    "userAssignedIdentities": null,
    "userAssignedIdentityExceptions": null
  },
  "powerState": {
    "code": "Running"
  },
  "privateFqdn": null,
  "privateLinkResources": null,
  "provisioningState": "Succeeded",
  "publicNetworkAccess": null,
  "resourceGroup": "[REDACTED]",
  "securityProfile": {
    "azureKeyVaultKms": null,
    "customCaTrustCertificates": null,
    "defender": null,
    "imageCleaner": null,
    "nodeRestriction": null,
    "workloadIdentity": null
  },
  "servicePrincipalProfile": {
    "clientId": "msi"
  },
  "sku": {
    "name": "Basic",
    "tier": "Free"
  },
  "storageProfile": {
    "blobCsiDriver": null,
    "diskCsiDriver": {
      "enabled": true,
      "version": "v1"
    },
    "fileCsiDriver": {
      "enabled": true
    },
    "snapshotController": {
      "enabled": true
    }
  },
  "systemData": null,
  "type": "Microsoft.ContainerService/ManagedClusters",
  "windowsProfile": {
    "adminPassword": null,
    "adminUsername": "[REDACTED]",
    "enableCsiProxy": true,
    "gmsaProfile": {
      "dnsServer": "",
      "enabled": true,
      "rootDomainName": ""
    },
    "licenseType": null
  },
  "workloadAutoScalerProfile": {
    "keda": null,
    "verticalPodAutoscaler": null
  }
}

yonzhan commented 2 years ago

route to CXP team

navba-MSFT commented 2 years ago

@m42jbz Thanks for reaching out to us and reporting this issue. We are looking into this issue and we will provide an update.

navba-MSFT commented 2 years ago

@m42jbz I am using the same version of AzCLI and aks-previewextension as your environment. But I am unable to reproduce this issue on my existing AKS cluster. I ran the below commands in order:

az feature register --name AKS-KedaPreview --namespace Microsoft.ContainerService az provider register --namespace Microsoft.ContainerService

az aks update --enable-keda --name MyAKSName --resource-group MyAKSRG

Query to check if Keda is enabled:

az aks show -g "MyAKSRG" --name MyAKSName --query "workloadAutoScalerProfile.keda.enabled" true

Could you please confirm if you are still facing the same issue ? Awaiting your reply.

m42jbz commented 2 years ago

The issue is still the same. No change. The output of az aks show -g ... -n ... --query "workloadAutoScalerProfile":

{
  "keda": null,
  "verticalPodAutoscaler": null
}

navba-MSFT commented 2 years ago

@m42jbz Thanks for getting back. We are looking deeper into this. We will provide an update soon.

Azure / azure-cli