search

Azure / azure-powershell

Microsoft Azure PowerShell
Other
4.26k stars 3.87k forks source link

Set-AzAksCluster with -AcrNameToAttach parameter not work on AKS clusters using managed identity #23812

Open gsxmax opened 11 months ago

gsxmax commented 11 months ago

Description

When using powershell command to attach the ACR for AKS cluster which are using managed identity, it would not work. It would try to find the SP in local enviorment first, if not exist, it would try to create a new one. However, actually it should grant the "ACRpull" role to the kubelet identity of the AKS cluster.

Issue script & Debug output

PS /home/max>> Set-AzAksCluster -Name testworkload -ResourceGroupName test -AcrNameToAttach maxdb 
DEBUG: 2:57:26 AM - SetAzureRmAks begin processing with ParameterSet 'defaultParameterSet'.
DEBUG: 2:57:26 AM - using account id 'MSI@50342'...
DEBUG: 2:57:26 AM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarning], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: [Common.Authentication]: Authenticating using Account: 'MSI@50342', environment: 'AzureCloud', tenant: 'xxxxxxxx-xxxxxxxx-2d7cd011db47'
DEBUG: 2:57:26 AM - [ManagedServiceIdentityAuthenticator] Calling ManagedIdentityCredential.GetTokenAsync - TenantId:'xxxxxxxx-xxxxxxxx-2d7cd011db47', Scopes:'https://management.core.windows.net/', UserId:''
DEBUG: ManagedIdentityCredential.GetToken invoked. Scopes: [ https://management.core.windows.net/ ] ParentRequestId: 
DEBUG: Request [584ca622-737c-4f54-9d71-a20db67d0d4b] POST http://localhost:50342/oauth2/token
Content-Type:application/x-www-form-urlencoded
Metadata:REDACTED
x-ms-client-request-id:584ca622-737c-4f54-9d71-a20db67d0d4b
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.6.1 (.NET 8.0.0; CBL-Mariner/Linux)
client assembly: Azure.Identity
DEBUG: Response [584ca622-737c-4f54-9d71-a20db67d0d4b] 200 OK (00.0s)
X-Powered-By:REDACTED
ETag:W/"981-L/+IFd9A7ccgHelADH6yYghwPUs"
Date:Fri, 22 Dec 2023 02:57:26 GMT
Connection:keep-alive
Keep-Alive:REDACTED
Content-Type:application/json; charset=utf-8
Content-Length:2433

DEBUG: ManagedIdentityCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net/ ] ParentRequestId:  ExpiresOn: 2023-12-22T03:59:52.0000000+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'xxxxxxxx-xxxxxxxx-2d7cd011db47', UserId: 'MSI@50342'
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/xxxxxxxx-xxxxxxxx/resourceGroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload?api-version=2023-04-01

Headers:
Accept-Language               : en-US
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T025727Z:75b258e9-eefd-48e0-8434-cb92b67031a2
x-ms-ratelimit-remaining-subscription-reads: 11997
x-ms-correlation-request-id   : 75b258e9-eefd-48e0-8434-cb92b67031a2
x-ms-request-id               : 4cf573da-7f72-4535-854b-51e9a45cf6a9
Strict-Transport-Security     : max-age=31536000; includeSubDomains
X-Content-Type-Options        : nosniff
Date                          : Fri, 22 Dec 2023 02:57:27 GMT

Body:
{
  "id": "/subscriptions/xxxxxxxx-xxxxxxxx/resourcegroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload",
  "location": "canadaeast",
  "name": "testworkload",
  "type": "Microsoft.ContainerService/ManagedClusters",
  "properties": {
    "provisioningState": "Succeeded",
    "powerState": {
      "code": "Running"
    },
    "kubernetesVersion": "1.26.10",
    "currentKubernetesVersion": "1.26.10",
    "dnsPrefix": "testworkload-dns",
    "fqdn": "testworkload-dns-h3jojj3y.hcp.canadaeast.azmk8s.io",
    "azurePortalFQDN": "testworkload-dns-h3jojj3y.portal.hcp.canadaeast.azmk8s.io",
    "agentPoolProfiles": [
      {
        "name": "agentpool",
        "count": 2,
        "vmSize": "Standard_DS2_v2",
        "osDiskSizeGB": 128,
        "osDiskType": "Managed",
        "kubeletDiskType": "OS",
        "maxPods": 110,
        "type": "VirtualMachineScaleSets",
        "maxCount": 5,
        "minCount": 2,
        "enableAutoScaling": true,
        "provisioningState": "Succeeded",
        "powerState": {
          "code": "Running"
        },
        "orchestratorVersion": "1.26.10",
        "currentOrchestratorVersion": "1.26.10",
        "enableNodePublicIP": false,
        "mode": "System",
        "osType": "Linux",
        "osSKU": "Ubuntu",
        "nodeImageVersion": "AKSUbuntu-2204gen2containerd-202312.06.0",
        "enableFIPS": false
      }
    ],
    "servicePrincipalProfile": {
      "clientId": "msi"
    },
    "addonProfiles": {
      "azureKeyvaultSecretsProvider": {
        "enabled": false,
        "config": null
      },
      "azurepolicy": {
        "enabled": false,
        "config": null
      }
    },
    "nodeResourceGroup": "MC_test_testworkload_canadaeast",
    "enableRBAC": true,
    "supportPlan": "KubernetesOfficial",
    "networkProfile": {
      "networkPlugin": "kubenet",
      "networkPolicy": "calico",
      "loadBalancerSku": "Standard",
      "loadBalancerProfile": {
        "managedOutboundIPs": {
          "count": 1
        },
        "effectiveOutboundIPs": [
          {
            "id": "/subscriptions/xxxxxxxx-xxxxxxxx/resourceGroups/MC_test_testworkload_canadaeast/providers/Microsoft.Network/publicIPAddresses/3e0d16f0-d311-4b62-a726-3073ca973d5e"
          }
        ]
      },
      "podCidr": "10.244.0.0/16",
      "serviceCidr": "10.0.0.0/16",
      "dnsServiceIP": "10.0.0.10",
      "outboundType": "loadBalancer",
      "podCidrs": [
        "10.244.0.0/16"
      ],
      "serviceCidrs": [
        "10.0.0.0/16"
      ],
      "ipFamilies": [
        "IPv4"
      ]
    },
    "maxAgentPools": 100,
    "identityProfile": {
      "kubeletidentity": {
        "resourceId": "/subscriptions/xxxxxxxx-xxxxxxxx/resourcegroups/MC_test_testworkload_canadaeast/providers/Microsoft.ManagedIdentity/userAssignedIdentities/testworkload-agentpool",
        "clientId": "4f9fe12e-ed17-45b6-bf7e-5f519c7cd096",
        "objectId": "fdaf2874-e45d-4b35-a70c-17eba6bd3be0"
      }
    },
    "autoScalerProfile": {
      "balance-similar-node-groups": "false",
      "expander": "random",
      "max-empty-bulk-delete": "10",
      "max-graceful-termination-sec": "600",
      "max-node-provision-time": "15m",
      "max-total-unready-percentage": "45",
      "new-pod-scale-up-delay": "0s",
      "ok-total-unready-count": "3",
      "scale-down-delay-after-add": "10m",
      "scale-down-delay-after-delete": "10s",
      "scale-down-delay-after-failure": "3m",
      "scale-down-unneeded-time": "10m",
      "scale-down-unready-time": "20m",
      "scale-down-utilization-threshold": "0.5",
      "scan-interval": "10s",
      "skip-nodes-with-local-storage": "false",
      "skip-nodes-with-system-pods": "true"
    },
    "autoUpgradeProfile": {
      "upgradeChannel": "patch"
    },
    "disableLocalAccounts": false,
    "securityProfile": {
      "workloadIdentity": {
        "enabled": true
      }
    },
    "storageProfile": {
      "diskCSIDriver": {
        "enabled": true
      },
      "fileCSIDriver": {
        "enabled": true
      },
      "snapshotController": {
        "enabled": true
      }
    },
    "oidcIssuerProfile": {
      "enabled": true,
      "issuerURL": "https://canadaeast.oic.prod-aks.azure.com/xxxxxxxx-xxxxxxxx-2d7cd011db47/edd8d595-5ec2-4231-8e6c-8346cc21e9ab/"
    },
    "workloadAutoScalerProfile": {}
  },
  "identity": {
    "type": "SystemAssigned",
    "principalId": "ba473d26-b261-433b-90a7-8a56a842b646",
    "tenantId": "xxxxxxxx-xxxxxxxx-2d7cd011db47"
  },
  "sku": {
    "name": "Base",
    "tier": "Free"
  }
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/xxxxxxxx-xxxxxxxx/resourceGroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload?api-version=2023-04-01

Headers:
Accept-Language               : en-US
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T025728Z:47689dba-b10a-46ce-a201-7b499a17d2d3
x-ms-ratelimit-remaining-subscription-reads: 11996
x-ms-correlation-request-id   : 47689dba-b10a-46ce-a201-7b499a17d2d3
x-ms-request-id               : d01a568f-3f0d-4eb3-98e6-fddd85d5722b
Strict-Transport-Security     : max-age=31536000; includeSubDomains
X-Content-Type-Options        : nosniff
Date                          : Fri, 22 Dec 2023 02:57:28 GMT

Body:
{
  "id": "/subscriptions/xxxxxxxx-xxxxxxxx/resourcegroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload",
  "location": "canadaeast",
  "name": "testworkload",
  "type": "Microsoft.ContainerService/ManagedClusters",
  "properties": {
    "provisioningState": "Succeeded",
    "powerState": {
      "code": "Running"
    },
    "kubernetesVersion": "1.26.10",
    "currentKubernetesVersion": "1.26.10",
    "dnsPrefix": "testworkload-dns",
    "fqdn": "testworkload-dns-h3jojj3y.hcp.canadaeast.azmk8s.io",
    "azurePortalFQDN": "testworkload-dns-h3jojj3y.portal.hcp.canadaeast.azmk8s.io",
    "agentPoolProfiles": [
      {
        "name": "agentpool",
        "count": 2,
        "vmSize": "Standard_DS2_v2",
        "osDiskSizeGB": 128,
        "osDiskType": "Managed",
        "kubeletDiskType": "OS",
        "maxPods": 110,
        "type": "VirtualMachineScaleSets",
        "maxCount": 5,
        "minCount": 2,
        "enableAutoScaling": true,
        "provisioningState": "Succeeded",
        "powerState": {
          "code": "Running"
        },
        "orchestratorVersion": "1.26.10",
        "currentOrchestratorVersion": "1.26.10",
        "enableNodePublicIP": false,
        "mode": "System",
        "osType": "Linux",
        "osSKU": "Ubuntu",
        "nodeImageVersion": "AKSUbuntu-2204gen2containerd-202312.06.0",
        "enableFIPS": false
      }
    ],
    "servicePrincipalProfile": {
      "clientId": "msi"
    },
    "addonProfiles": {
      "azureKeyvaultSecretsProvider": {
        "enabled": false,
        "config": null
      },
      "azurepolicy": {
        "enabled": false,
        "config": null
      }
    },
    "nodeResourceGroup": "MC_test_testworkload_canadaeast",
    "enableRBAC": true,
    "supportPlan": "KubernetesOfficial",
    "networkProfile": {
      "networkPlugin": "kubenet",
      "networkPolicy": "calico",
      "loadBalancerSku": "Standard",
      "loadBalancerProfile": {
        "managedOutboundIPs": {
          "count": 1
        },
        "effectiveOutboundIPs": [
          {
            "id": "/subscriptions/xxxxxxxx-xxxxxxxx/resourceGroups/MC_test_testworkload_canadaeast/providers/Microsoft.Network/publicIPAddresses/3e0d16f0-d311-4b62-a726-3073ca973d5e"
          }
        ]
      },
      "podCidr": "10.244.0.0/16",
      "serviceCidr": "10.0.0.0/16",
      "dnsServiceIP": "10.0.0.10",
      "outboundType": "loadBalancer",
      "podCidrs": [
        "10.244.0.0/16"
      ],
      "serviceCidrs": [
        "10.0.0.0/16"
      ],
      "ipFamilies": [
        "IPv4"
      ]
    },
    "maxAgentPools": 100,
    "identityProfile": {
      "kubeletidentity": {
        "resourceId": "/subscriptions/xxxxxxxx-xxxxxxxx/resourcegroups/MC_test_testworkload_canadaeast/providers/Microsoft.ManagedIdentity/userAssignedIdentities/testworkload-agentpool",
        "clientId": "4f9fe12e-ed17-45b6-bf7e-5f519c7cd096",
        "objectId": "fdaf2874-e45d-4b35-a70c-17eba6bd3be0"
      }
    },
    "autoScalerProfile": {
      "balance-similar-node-groups": "false",
      "expander": "random",
      "max-empty-bulk-delete": "10",
      "max-graceful-termination-sec": "600",
      "max-node-provision-time": "15m",
      "max-total-unready-percentage": "45",
      "new-pod-scale-up-delay": "0s",
      "ok-total-unready-count": "3",
      "scale-down-delay-after-add": "10m",
      "scale-down-delay-after-delete": "10s",
      "scale-down-delay-after-failure": "3m",
      "scale-down-unneeded-time": "10m",
      "scale-down-unready-time": "20m",
      "scale-down-utilization-threshold": "0.5",
      "scan-interval": "10s",
      "skip-nodes-with-local-storage": "false",
      "skip-nodes-with-system-pods": "true"
    },
    "autoUpgradeProfile": {
      "upgradeChannel": "patch"
    },
    "disableLocalAccounts": false,
    "securityProfile": {
      "workloadIdentity": {
        "enabled": true
      }
    },
    "storageProfile": {
      "diskCSIDriver": {
        "enabled": true
      },
      "fileCSIDriver": {
        "enabled": true
      },
      "snapshotController": {
        "enabled": true
      }
    },
    "oidcIssuerProfile": {
      "enabled": true,
      "issuerURL": "https://canadaeast.oic.prod-aks.azure.com/xxxxxxxx-xxxxxxxx-2d7cd011db47/edd8d595-5ec2-4231-8e6c-8346cc21e9ab/"
    },
    "workloadAutoScalerProfile": {}
  },
  "identity": {
    "type": "SystemAssigned",
    "principalId": "ba473d26-b261-433b-90a7-8a56a842b646",
    "tenantId": "xxxxxxxx-xxxxxxxx-2d7cd011db47"
  },
  "sku": {
    "name": "Base",
    "tier": "Free"
  }
}

WARNING: No Service Principal found in /home/max/.azure/acsServicePrincipal.json for current subscription xxxxxxxx-xxxxxxxx. Trying to create a new Service Principal with Contributor role for the subscription.
DE

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
PUT

Absolute Uri:
https://management.azure.com//subscriptions/xxxxxxxx-f2ecd6a53c42/resourceGroups/myRG/providers/Microsoft.ContainerRegistry/registries/maxdb/providers/Microsoft.Authorization/roleAssignments/c6cdc95d-ee9b-4b4e-9a49-829284f49397?api-version=2015-07-01

Headers:
Accept-Language               : en-US
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:
{
  "properties": {
    "roleDefinitionId": "/subscriptions/xxxxxxxx-f2ecd6a53c42/providers/Microsoft.Authorization/roleDefinitions/7f951dda-4ed3-4680-a7ca-43fe172d538d",
    "principalId": "f6f4c3dd-f411-4a42-9d62-aed53e1456b5"
  }
}

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
Created

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-request-id               : db2fff3a-adff-4e40-b2b2-72394bf83d55
X-Content-Type-Options        : nosniff
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-ratelimit-remaining-subscription-writes: 1193
x-ms-correlation-request-id   : 8c7a1b57-dc46-4026-900e-ee96df87f88a
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T025804Z:8c7a1b57-dc46-4026-900e-ee96df87f88a
Date                          : Fri, 22 Dec 2023 02:58:03 GMT

Body:
{
  "properties": {
    "roleDefinitionId": "/subscriptions/xxxxxxxx-f2ecd6a53c42/providers/Microsoft.Authorization/roleDefinitions/7f951dda-4ed3-4680-a7ca-43fe172d538d",
    "principalId": "f6f4c3dd-f411-4a42-9d62-aed53e1456b5",
    "scope": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourceGroups/myRG/providers/Microsoft.ContainerRegistry/registries/maxdb",
    "createdOn": "2023-12-22T02:58:01.1372376Z",
    "updatedOn": "2023-12-22T02:58:01.4622434Z",
    "createdBy": null,
    "updatedBy": "9d965875-bc32-4fed-8ae5-9b01dcaccbe9"
  },
  "id": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourceGroups/myRG/providers/Microsoft.ContainerRegistry/registries/maxdb/providers/Microsoft.Authorization/roleAssignments/c6cdc95d-ee9b-4b4e-9a49-829284f49397",
  "type": "Microsoft.Authorization/roleAssignments",
  "name": "c6cdc95d-ee9b-4b4e-9a49-829284f49397"
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
PUT

Absolute Uri:
https://management.azure.com/subscriptions/xxxxxxxx-f2ecd6a53c42/resourceGroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload?api-version=2023-04-01

Headers:
Accept-Language               : en-US
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:
{
  "sku": {
    "name": "Base",
    "tier": "Free"
  },
  "identity": {
    "type": "SystemAssigned"
  },
  "properties": {
    "kubernetesVersion": "1.26.10",
    "dnsPrefix": "testworkload-dns",
    "agentPoolProfiles": [
      {
        "name": "agentpool",
        "count": 2,
        "vmSize": "Standard_DS2_v2",
        "osDiskSizeGB": 128,
        "osDiskType": "Managed",
        "kubeletDiskType": "OS",
        "maxPods": 110,
        "osType": "Linux",
        "osSKU": "Ubuntu",
        "maxCount": 5,
        "minCount": 2,
        "enableAutoScaling": true,
        "type": "VirtualMachineScaleSets",
        "mode": "System",
        "powerState": {
          "code": "Running"
        },
        "enableNodePublicIP": false,
        "enableFIPS": false
      }
    ],
    "servicePrincipalProfile": {
      "clientId": "msi"
    },
    "addonProfiles": {
      "azureKeyvaultSecretsProvider": {
        "enabled": false
      },
      "azurepolicy": {
        "enabled": false
      }
    },
    "oidcIssuerProfile": {
      "enabled": true
    },
    "nodeResourceGroup": "MC_test_testworkload_canadaeast",
    "enableRBAC": true,
    "supportPlan": "KubernetesOfficial",
    "networkProfile": {
      "networkPlugin": "kubenet",
      "networkPolicy": "calico",
      "podCidr": "10.244.0.0/16",
      "serviceCidr": "10.0.0.0/16",
      "dnsServiceIP": "10.0.0.10",
      "outboundType": "loadBalancer",
      "loadBalancerSku": "Standard",
      "loadBalancerProfile": {
        "managedOutboundIPs": {
          "count": 1
        },
        "effectiveOutboundIPs": [
          {
            "id": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourceGroups/MC_test_testworkload_canadaeast/providers/Microsoft.Network/publicIPAddresses/3e0d16f0-d311-4b62-a726-3073ca973d5e"
          }
        ]
      },
      "podCidrs": [
        "10.244.0.0/16"
      ],
      "serviceCidrs": [
        "10.0.0.0/16"
      ],
      "ipFamilies": [
        "IPv4"
      ]
    },
    "autoUpgradeProfile": {
      "upgradeChannel": "patch"
    },
    "autoScalerProfile": {
      "balance-similar-node-groups": "false",
      "expander": "random",
      "max-empty-bulk-delete": "10",
      "max-graceful-termination-sec": "600",
      "max-node-provision-time": "15m",
      "max-total-unready-percentage": "45",
      "new-pod-scale-up-delay": "0s",
      "ok-total-unready-count": "3",
      "scan-interval": "10s",
      "scale-down-delay-after-add": "10m",
      "scale-down-delay-after-delete": "10s",
      "scale-down-delay-after-failure": "3m",
      "scale-down-unneeded-time": "10m",
      "scale-down-unready-time": "20m",
      "scale-down-utilization-threshold": "0.5",
      "skip-nodes-with-local-storage": "false",
      "skip-nodes-with-system-pods": "true"
    },
    "identityProfile": {
      "kubeletidentity": {
        "resourceId": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourcegroups/MC_test_testworkload_canadaeast/providers/Microsoft.ManagedIdentity/userAssignedIdentities/testworkload-agentpool",
        "clientId": "4f9fe12e-ed17-45b6-bf7e-5f519c7cd096",
        "objectId": "fdaf2874-e45d-4b35-a70c-17eba6bd3be0"
      }
    },
    "disableLocalAccounts": false,
    "securityProfile": {
      "workloadIdentity": {
        "enabled": true
      }
    },
    "storageProfile": {
      "diskCSIDriver": {
        "enabled": true
      },
      "fileCSIDriver": {
        "enabled": true
      },
      "snapshotController": {
        "enabled": true
      }
    },
    "workloadAutoScalerProfile": {}
  },
  "location": "canadaeast"
}

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T025809Z:ac884532-aa1d-4fd7-bc83-f0e913844397
x-ms-ratelimit-remaining-subscription-writes: 1199
Azure-AsyncOperation          : https://management.azure.com/subscriptions/xxxxxxxx-f2ecd6a53c42/providers/Microsoft.ContainerService/locations/canadaeast/operations/186f6061-4269-4aaa-936d-523481f042b8?api-version=2016-03-30&t=638388106895216045&c=MIIHHjCCBgagAwIBAgITOgI9LlF5jGEEQe1CmgAEAj0uUTANBgkqhkiG9w0BAQsFADBEMRMwEQYKCZImiZPyLGQBGRYDR0JMMRMwEQYKCZImiZPyLGQBGRYDQU1FMRgwFgYDVQQDEw9BTUUgSU5GUkEgQ0EgMDEwHhcNMjMxMTAxMjE0OTQ5WhcNMjQxMDI2MjE0OTQ5WjBAMT4wPAYDVQQDEzVhc3luY29wZXJhdGlvbnNpZ25pbmdjZXJ0aWZpY2F0ZS5tYW5hZ2VtZW50LmF6dXJlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMKt_zl2yRFDxIUmqoEA-B7WoRkSqscgwk6CGqDbwEVJS3oTI4Gk-ukQ6El3GQJVthiuwZGw4jwvTDyVWC8mQH2Zni6X1ocTXrNNa6VFJnvzeEn-HpmPATuII8AWtdtUnBvGZGAwgspvH8iZZCiHDowHCWhjVsU6_ipCsmzQeH3oRJI4AwUS_nCVA7U9WPdgIyQp6v5QPzcYCO_zIbNW4y_Cc3phfPEQvPvtSKhuijt68F5WlUY6Ps23ksosYG5VTx7z6skGG9arA5UKUzbbEO8lxzda3OFPZdG7OHe8-VMXxJjtSanZSSbdLgYBRB9m9PGr6voKtML9W5rEukE-lEUCAwEAAaOCBAswggQHMCcGCSsGAQQBgjcVCgQaMBgwCgYIKwYBBQUHAwEwCgYIKwYBBQUHAwIwPQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIhpDjDYTVtHiE8Ys-hZvdFs6dEoFggvX2K4Py0SACAWQCAQowggHaBggrBgEFBQcBAQSCAcwwggHIMGYGCCsGAQUFBzAChlpodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpaW5mcmEvQ2VydHMvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmwxLmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MFYGCCsGAQUFBzAChkpodHRwOi8vY3JsMi5hbWUuZ2JsL2FpYS9CWTJQS0lJTlRDQTAxLkFNRS5HQkxfQU1FJTIwSU5GUkElMjBDQSUyMDAxKDQpLmNydDBWBggrBgEFBQcwAoZKaHR0cDovL2NybDMuYW1lLmdibC9haWEvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmw0LmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MB0GA1UdDgQWBBRYUmiD-EjeFFNZByiKICN9p1JofzAOBgNVHQ8BAf8EBAMCBaAwggE1BgNVHR8EggEsMIIBKDCCASSgggEgoIIBHIZCaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraWluZnJhL0NSTC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMS5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMi5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMy5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsNC5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JsMBcGA1UdIAQQMA4wDAYKKwYBBAGCN3sBATAfBgNVHSMEGDAWgBTl2Ztn_PjsurvwwKidileIud8-YzAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKRwStAdrsy7tgEfmtXPZXIwgdXOYwSx2Kqzlhq8b6Ptid4V3vhJvDV0WLS7d72KIMUwZiBnfPHhMmsdkLOSpNSZYC23g3QJSBWvoS86Rpkeru5bEyQmv1gucF_MyaFaUXUd4b7xnvCkv-BBNO8m1lvFAowB2odz9esggf82yeOVBx0yEK_NNOK4l36XvRjvIqemSLZOSi1wow_wn_ovhAEPpSSN7UTahPtz7J12vhP1LuZ4Rvn_hNx48ncw445nMBnMVmcuuakaKDC6_sAcs5lg9jq2yHSaIHRw4xcoS5XwvIykGPMUyAD-dXWFih_XFOKh7ea50hObFhghf3Dtvlo&s=Dgt2QCbawbcFWOt9zWt1hj_3DStmg5bsbaznD9MzuUaVCPHJfp7y4o1u6huWIC7u7oenmATkuVsyWV1yEtXrycGPL0dOj_af6O5FQGy27nUDDOcDA1Gxv9cUfUrslRkd6H2EF8DqyoEpjLM1iwwl5H1MGOQogJ8y5NM2gVol1N3O18cZ7IZXzi4ddKmbL37iahdQhjoIqgIofpezdIvvW_pbxk0S10Y6TTOT8-CVOrBHZ2qVss_1wY7qfSlWMUQPZGBrmu6BdPUTCoQ5mSr-yA4rzrhQkMSuqWEch6GqIBpf425hIWO9TY8SIte9Cjf5GK_AwYIE-B98GrI1tSIlsA&h=CrBmGd1b7XJZiJkAsLonXmsv5P0Rs_rQRKDragy6rjk
x-ms-correlation-request-id   : ac884532-aa1d-4fd7-bc83-f0e913844397
x-ms-request-id               : 186f6061-4269-4aaa-936d-523481f042b8
Strict-Transport-Security     : max-age=31536000; includeSubDomains
X-Content-Type-Options        : nosniff
Date                          : Fri, 22 Dec 2023 02:58:08 GMT

Body:
{
  "id": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourcegroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload",
  "location": "canadaeast",
  "name": "testworkload",
  "type": "Microsoft.ContainerService/ManagedClusters",
  "properties": {
    "provisioningState": "Updating",
    "powerState": {
      "code": "Running"
    },
    "kubernetesVersion": "1.26.10",
    "currentKubernetesVersion": "1.26.10",
    "dnsPrefix": "testworkload-dns",
    "fqdn": "testworkload-dns-h3jojj3y.hcp.canadaeast.azmk8s.io",
    "azurePortalFQDN": "testworkload-dns-h3jojj3y.portal.hcp.canadaeast.azmk8s.io",
    "agentPoolProfiles": [
      {
        "name": "agentpool",
        "count": 2,
        "vmSize": "Standard_DS2_v2",
        "osDiskSizeGB": 128,
        "osDiskType": "Managed",
        "kubeletDiskType": "OS",
        "maxPods": 110,
        "type": "VirtualMachineScaleSets",
        "maxCount": 5,
        "minCount": 2,
        "enableAutoScaling": true,
        "provisioningState": "Updating",
        "powerState": {
          "code": "Running"
        },
        "orchestratorVersion": "1.26.10",
        "currentOrchestratorVersion": "1.26.10",
        "enableNodePublicIP": false,
        "nodeLabels": {},
        "mode": "System",
        "osType": "Linux",
        "osSKU": "Ubuntu",
        "nodeImageVersion": "AKSUbuntu-2204gen2containerd-202312.06.0",
        "enableFIPS": false
      }
    ],
    "servicePrincipalProfile": {
      "clientId": "msi"
    },
    "addonProfiles": {
      "azureKeyvaultSecretsProvider": {
        "enabled": false,
        "config": null
      },
      "azurepolicy": {
        "enabled": false,
        "config": null
      }
    },
    "nodeResourceGroup": "MC_test_testworkload_canadaeast",
    "enableRBAC": true,
    "supportPlan": "KubernetesOfficial",
    "networkProfile": {
      "networkPlugin": "kubenet",
      "networkPolicy": "calico",
      "loadBalancerSku": "Standard",
      "loadBalancerProfile": {
        "managedOutboundIPs": {
          "count": 1
        },
        "effectiveOutboundIPs": [
          {
            "id": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourceGroups/MC_test_testworkload_canadaeast/providers/Microsoft.Network/publicIPAddresses/3e0d16f0-d311-4b62-a726-3073ca973d5e"
          }
        ]
      },
      "podCidr": "10.244.0.0/16",
      "serviceCidr": "10.0.0.0/16",
      "dnsServiceIP": "10.0.0.10",
      "outboundType": "loadBalancer",
      "podCidrs": [
        "10.244.0.0/16"
      ],
      "serviceCidrs": [
        "10.0.0.0/16"
      ],
      "ipFamilies": [
        "IPv4"
      ]
    },
    "maxAgentPools": 100,
    "identityProfile": {
      "kubeletidentity": {
        "resourceId": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourcegroups/MC_test_testworkload_canadaeast/providers/Microsoft.ManagedIdentity/userAssignedIdentities/testworkload-agentpool",
        "clientId": "4f9fe12e-ed17-45b6-bf7e-5f519c7cd096",
        "objectId": "fdaf2874-e45d-4b35-a70c-17eba6bd3be0"
      }
    },
    "autoScalerProfile": {
      "balance-similar-node-groups": "false",
      "expander": "random",
      "max-empty-bulk-delete": "10",
      "max-graceful-termination-sec": "600",
      "max-node-provision-time": "15m",
      "max-total-unready-percentage": "45",
      "new-pod-scale-up-delay": "0s",
      "ok-total-unready-count": "3",
      "scale-down-delay-after-add": "10m",
      "scale-down-delay-after-delete": "10s",
      "scale-down-delay-after-failure": "3m",
      "scale-down-unneeded-time": "10m",
      "scale-down-unready-time": "20m",
      "scale-down-utilization-threshold": "0.5",
      "scan-interval": "10s",
      "skip-nodes-with-local-storage": "false",
      "skip-nodes-with-system-pods": "true"
    },
    "autoUpgradeProfile": {
      "upgradeChannel": "patch"
    },
    "disableLocalAccounts": false,
    "securityProfile": {
      "workloadIdentity": {
        "enabled": true
      }
    },
    "storageProfile": {
      "diskCSIDriver": {
        "enabled": true
      },
      "fileCSIDriver": {
        "enabled": true
      },
      "snapshotController": {
        "enabled": true
      }
    },
    "oidcIssuerProfile": {
      "enabled": true,
      "issuerURL": "https://canadaeast.oic.prod-aks.azure.com/xxxxxxxx/edd8d595-5ec2-4231-8e6c-8346cc21e9ab/"
    },
    "workloadAutoScalerProfile": {}
  },
  "identity": {
    "type": "SystemAssigned",
    "principalId": "ba473d26-b261-433b-90a7-8a56a842b646",
    "tenantId": "xxxxxxxx"
  },
  "sku": {
    "name": "Base",
    "tier": "Free"
  }
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/xxxxxxxx-f2ecd6a53c42/providers/Microsoft.ContainerService/locations/canadaeast/operations/186f6061-4269-4aaa-936d-523481f042b8?api-version=2016-03-30&t=638388106895216045&c=MIIHHjCCBgagAwIBAgITOgI9LlF5jGEEQe1CmgAEAj0uUTANBgkqhkiG9w0BAQsFADBEMRMwEQYKCZImiZPyLGQBGRYDR0JMMRMwEQYKCZImiZPyLGQBGRYDQU1FMRgwFgYDVQQDEw9BTUUgSU5GUkEgQ0EgMDEwHhcNMjMxMTAxMjE0OTQ5WhcNMjQxMDI2MjE0OTQ5WjBAMT4wPAYDVQQDEzVhc3luY29wZXJhdGlvbnNpZ25pbmdjZXJ0aWZpY2F0ZS5tYW5hZ2VtZW50LmF6dXJlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMKt_zl2yRFDxIUmqoEA-B7WoRkSqscgwk6CGqDbwEVJS3oTI4Gk-ukQ6El3GQJVthiuwZGw4jwvTDyVWC8mQH2Zni6X1ocTXrNNa6VFJnvzeEn-HpmPATuII8AWtdtUnBvGZGAwgspvH8iZZCiHDowHCWhjVsU6_ipCsmzQeH3oRJI4AwUS_nCVA7U9WPdgIyQp6v5QPzcYCO_zIbNW4y_Cc3phfPEQvPvtSKhuijt68F5WlUY6Ps23ksosYG5VTx7z6skGG9arA5UKUzbbEO8lxzda3OFPZdG7OHe8-VMXxJjtSanZSSbdLgYBRB9m9PGr6voKtML9W5rEukE-lEUCAwEAAaOCBAswggQHMCcGCSsGAQQBgjcVCgQaMBgwCgYIKwYBBQUHAwEwCgYIKwYBBQUHAwIwPQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIhpDjDYTVtHiE8Ys-hZvdFs6dEoFggvX2K4Py0SACAWQCAQowggHaBggrBgEFBQcBAQSCAcwwggHIMGYGCCsGAQUFBzAChlpodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpaW5mcmEvQ2VydHMvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmwxLmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MFYGCCsGAQUFBzAChkpodHRwOi8vY3JsMi5hbWUuZ2JsL2FpYS9CWTJQS0lJTlRDQTAxLkFNRS5HQkxfQU1FJTIwSU5GUkElMjBDQSUyMDAxKDQpLmNydDBWBggrBgEFBQcwAoZKaHR0cDovL2NybDMuYW1lLmdibC9haWEvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmw0LmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MB0GA1UdDgQWBBRYUmiD-EjeFFNZByiKICN9p1JofzAOBgNVHQ8BAf8EBAMCBaAwggE1BgNVHR8EggEsMIIBKDCCASSgggEgoIIBHIZCaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraWluZnJhL0NSTC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMS5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMi5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMy5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsNC5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JsMBcGA1UdIAQQMA4wDAYKKwYBBAGCN3sBATAfBgNVHSMEGDAWgBTl2Ztn_PjsurvwwKidileIud8-YzAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKRwStAdrsy7tgEfmtXPZXIwgdXOYwSx2Kqzlhq8b6Ptid4V3vhJvDV0WLS7d72KIMUwZiBnfPHhMmsdkLOSpNSZYC23g3QJSBWvoS86Rpkeru5bEyQmv1gucF_MyaFaUXUd4b7xnvCkv-BBNO8m1lvFAowB2odz9esggf82yeOVBx0yEK_NNOK4l36XvRjvIqemSLZOSi1wow_wn_ovhAEPpSSN7UTahPtz7J12vhP1LuZ4Rvn_hNx48ncw445nMBnMVmcuuakaKDC6_sAcs5lg9jq2yHSaIHRw4xcoS5XwvIykGPMUyAD-dXWFih_XFOKh7ea50hObFhghf3Dtvlo&s=Dgt2QCbawbcFWOt9zWt1hj_3DStmg5bsbaznD9MzuUaVCPHJfp7y4o1u6huWIC7u7oenmATkuVsyWV1yEtXrycGPL0dOj_af6O5FQGy27nUDDOcDA1Gxv9cUfUrslRkd6H2EF8DqyoEpjLM1iwwl5H1MGOQogJ8y5NM2gVol1N3O18cZ7IZXzi4ddKmbL37iahdQhjoIqgIofpezdIvvW_pbxk0S10Y6TTOT8-CVOrBHZ2qVss_1wY7qfSlWMUQPZGBrmu6BdPUTCoQ5mSr-yA4rzrhQkMSuqWEch6GqIBpf425hIWO9TY8SIte9Cjf5GK_AwYIE-B98GrI1tSIlsA&h=CrBmGd1b7XJZiJkAsLonXmsv5P0Rs_rQRKDragy6rjk

Headers:
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-ratelimit-remaining-subscription-reads: 11995
x-ms-correlation-request-id   : ee461bec-70c2-4649-9392-2753a85a0367
x-ms-request-id               : 7486326e-747f-4712-979b-87706fe09108
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T025839Z:ee461bec-70c2-4649-9392-2753a85a0367
X-Content-Type-Options        : nosniff
Date                          : Fri, 22 Dec 2023 02:58:39 GMT

Body:
{
  "name": "186f6061-4269-4aaa-936d-523481f042b8",
  "status": "InProgress",
  "startTime": "2023-12-22T02:58:09.030451Z",
  "error": {
    "code": "NotLatestOperation",
    "message": "Cannot proceed with the operation. Either the operation has been preempted by another one, or the information needed by the operation failed to be saved (or hasn't been saved yet)."
  }
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/xxxxxxxx-f2ecd6a53c42/providers/Microsoft.ContainerService/locations/canadaeast/operations/186f6061-4269-4aaa-936d-523481f042b8?api-version=2016-03-30&t=638388106895216045&c=MIIHHjCCBgagAwIBAgITOgI9LlF5jGEEQe1CmgAEAj0uUTANBgkqhkiG9w0BAQsFADBEMRMwEQYKCZImiZPyLGQBGRYDR0JMMRMwEQYKCZImiZPyLGQBGRYDQU1FMRgwFgYDVQQDEw9BTUUgSU5GUkEgQ0EgMDEwHhcNMjMxMTAxMjE0OTQ5WhcNMjQxMDI2MjE0OTQ5WjBAMT4wPAYDVQQDEzVhc3luY29wZXJhdGlvbnNpZ25pbmdjZXJ0aWZpY2F0ZS5tYW5hZ2VtZW50LmF6dXJlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMKt_zl2yRFDxIUmqoEA-B7WoRkSqscgwk6CGqDbwEVJS3oTI4Gk-ukQ6El3GQJVthiuwZGw4jwvTDyVWC8mQH2Zni6X1ocTXrNNa6VFJnvzeEn-HpmPATuII8AWtdtUnBvGZGAwgspvH8iZZCiHDowHCWhjVsU6_ipCsmzQeH3oRJI4AwUS_nCVA7U9WPdgIyQp6v5QPzcYCO_zIbNW4y_Cc3phfPEQvPvtSKhuijt68F5WlUY6Ps23ksosYG5VTx7z6skGG9arA5UKUzbbEO8lxzda3OFPZdG7OHe8-VMXxJjtSanZSSbdLgYBRB9m9PGr6voKtML9W5rEukE-lEUCAwEAAaOCBAswggQHMCcGCSsGAQQBgjcVCgQaMBgwCgYIKwYBBQUHAwEwCgYIKwYBBQUHAwIwPQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIhpDjDYTVtHiE8Ys-hZvdFs6dEoFggvX2K4Py0SACAWQCAQowggHaBggrBgEFBQcBAQSCAcwwggHIMGYGCCsGAQUFBzAChlpodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpaW5mcmEvQ2VydHMvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmwxLmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MFYGCCsGAQUFBzAChkpodHRwOi8vY3JsMi5hbWUuZ2JsL2FpYS9CWTJQS0lJTlRDQTAxLkFNRS5HQkxfQU1FJTIwSU5GUkElMjBDQSUyMDAxKDQpLmNydDBWBggrBgEFBQcwAoZKaHR0cDovL2NybDMuYW1lLmdibC9haWEvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmw0LmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MB0GA1UdDgQWBBRYUmiD-EjeFFNZByiKICN9p1JofzAOBgNVHQ8BAf8EBAMCBaAwggE1BgNVHR8EggEsMIIBKDCCASSgggEgoIIBHIZCaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraWluZnJhL0NSTC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMS5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMi5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMy5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsNC5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JsMBcGA1UdIAQQMA4wDAYKKwYBBAGCN3sBATAfBgNVHSMEGDAWgBTl2Ztn_PjsurvwwKidileIud8-YzAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKRwStAdrsy7tgEfmtXPZXIwgdXOYwSx2Kqzlhq8b6Ptid4V3vhJvDV0WLS7d72KIMUwZiBnfPHhMmsdkLOSpNSZYC23g3QJSBWvoS86Rpkeru5bEyQmv1gucF_MyaFaUXUd4b7xnvCkv-BBNO8m1lvFAowB2odz9esggf82yeOVBx0yEK_NNOK4l36XvRjvIqemSLZOSi1wow_wn_ovhAEPpSSN7UTahPtz7J12vhP1LuZ4Rvn_hNx48ncw445nMBnMVmcuuakaKDC6_sAcs5lg9jq2yHSaIHRw4xcoS5XwvIykGPMUyAD-dXWFih_XFOKh7ea50hObFhghf3Dtvlo&s=Dgt2QCbawbcFWOt9zWt1hj_3DStmg5bsbaznD9MzuUaVCPHJfp7y4o1u6huWIC7u7oenmATkuVsyWV1yEtXrycGPL0dOj_af6O5FQGy27nUDDOcDA1Gxv9cUfUrslRkd6H2EF8DqyoEpjLM1iwwl5H1MGOQogJ8y5NM2gVol1N3O18cZ7IZXzi4ddKmbL37iahdQhjoIqgIofpezdIvvW_pbxk0S10Y6TTOT8-CVOrBHZ2qVss_1wY7qfSlWMUQPZGBrmu6BdPUTCoQ5mSr-yA4rzrhQkMSuqWEch6GqIBpf425hIWO9TY8SIte9Cjf5GK_AwYIE-B98GrI1tSIlsA&h=CrBmGd1b7XJZiJkAsLonXmsv5P0Rs_rQRKDragy6rjk

Headers:
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-ratelimit-remaining-subscription-reads: 11994
x-ms-correlation-request-id   : 1b6a561b-2194-48e9-abe6-91d2c3c3e432
x-ms-request-id               : 797f2ddb-8244-465c-b626-77259b38eea6
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T025910Z:1b6a561b-2194-48e9-abe6-91d2c3c3e432
X-Content-Type-Options        : nosniff
Date                          : Fri, 22 Dec 2023 02:59:09 GMT

Body:
{
  "name": "186f6061-4269-4aaa-936d-523481f042b8",
  "status": "InProgress",
  "startTime": "2023-12-22T02:58:09.030451Z",
  "error": {
    "code": "NotLatestOperation",
    "message": "Cannot proceed with the operation. Either the operation has been preempted by another one, or the information needed by the operation failed to be saved (or hasn't been saved yet)."
  }
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/xxxxxxxx-f2ecd6a53c42/providers/Microsoft.ContainerService/locations/canadaeast/operations/186f6061-4269-4aaa-936d-523481f042b8?api-version=2016-03-30&t=638388106895216045&c=MIIHHjCCBgagAwIBAgITOgI9LlF5jGEEQe1CmgAEAj0uUTANBgkqhkiG9w0BAQsFADBEMRMwEQYKCZImiZPyLGQBGRYDR0JMMRMwEQYKCZImiZPyLGQBGRYDQU1FMRgwFgYDVQQDEw9BTUUgSU5GUkEgQ0EgMDEwHhcNMjMxMTAxMjE0OTQ5WhcNMjQxMDI2MjE0OTQ5WjBAMT4wPAYDVQQDEzVhc3luY29wZXJhdGlvbnNpZ25pbmdjZXJ0aWZpY2F0ZS5tYW5hZ2VtZW50LmF6dXJlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMKt_zl2yRFDxIUmqoEA-B7WoRkSqscgwk6CGqDbwEVJS3oTI4Gk-ukQ6El3GQJVthiuwZGw4jwvTDyVWC8mQH2Zni6X1ocTXrNNa6VFJnvzeEn-HpmPATuII8AWtdtUnBvGZGAwgspvH8iZZCiHDowHCWhjVsU6_ipCsmzQeH3oRJI4AwUS_nCVA7U9WPdgIyQp6v5QPzcYCO_zIbNW4y_Cc3phfPEQvPvtSKhuijt68F5WlUY6Ps23ksosYG5VTx7z6skGG9arA5UKUzbbEO8lxzda3OFPZdG7OHe8-VMXxJjtSanZSSbdLgYBRB9m9PGr6voKtML9W5rEukE-lEUCAwEAAaOCBAswggQHMCcGCSsGAQQBgjcVCgQaMBgwCgYIKwYBBQUHAwEwCgYIKwYBBQUHAwIwPQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIhpDjDYTVtHiE8Ys-hZvdFs6dEoFggvX2K4Py0SACAWQCAQowggHaBggrBgEFBQcBAQSCAcwwggHIMGYGCCsGAQUFBzAChlpodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpaW5mcmEvQ2VydHMvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmwxLmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MFYGCCsGAQUFBzAChkpodHRwOi8vY3JsMi5hbWUuZ2JsL2FpYS9CWTJQS0lJTlRDQTAxLkFNRS5HQkxfQU1FJTIwSU5GUkElMjBDQSUyMDAxKDQpLmNydDBWBggrBgEFBQcwAoZKaHR0cDovL2NybDMuYW1lLmdibC9haWEvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmw0LmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MB0GA1UdDgQWBBRYUmiD-EjeFFNZByiKICN9p1JofzAOBgNVHQ8BAf8EBAMCBaAwggE1BgNVHR8EggEsMIIBKDCCASSgggEgoIIBHIZCaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraWluZnJhL0NSTC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMS5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMi5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMy5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsNC5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JsMBcGA1UdIAQQMA4wDAYKKwYBBAGCN3sBATAfBgNVHSMEGDAWgBTl2Ztn_PjsurvwwKidileIud8-YzAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKRwStAdrsy7tgEfmtXPZXIwgdXOYwSx2Kqzlhq8b6Ptid4V3vhJvDV0WLS7d72KIMUwZiBnfPHhMmsdkLOSpNSZYC23g3QJSBWvoS86Rpkeru5bEyQmv1gucF_MyaFaUXUd4b7xnvCkv-BBNO8m1lvFAowB2odz9esggf82yeOVBx0yEK_NNOK4l36XvRjvIqemSLZOSi1wow_wn_ovhAEPpSSN7UTahPtz7J12vhP1LuZ4Rvn_hNx48ncw445nMBnMVmcuuakaKDC6_sAcs5lg9jq2yHSaIHRw4xcoS5XwvIykGPMUyAD-dXWFih_XFOKh7ea50hObFhghf3Dtvlo&s=Dgt2QCbawbcFWOt9zWt1hj_3DStmg5bsbaznD9MzuUaVCPHJfp7y4o1u6huWIC7u7oenmATkuVsyWV1yEtXrycGPL0dOj_af6O5FQGy27nUDDOcDA1Gxv9cUfUrslRkd6H2EF8DqyoEpjLM1iwwl5H1MGOQogJ8y5NM2gVol1N3O18cZ7IZXzi4ddKmbL37iahdQhjoIqgIofpezdIvvW_pbxk0S10Y6TTOT8-CVOrBHZ2qVss_1wY7qfSlWMUQPZGBrmu6BdPUTCoQ5mSr-yA4rzrhQkMSuqWEch6GqIBpf425hIWO9TY8SIte9Cjf5GK_AwYIE-B98GrI1tSIlsA&h=CrBmGd1b7XJZiJkAsLonXmsv5P0Rs_rQRKDragy6rjk

Headers:
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-ratelimit-remaining-subscription-reads: 11993
x-ms-correlation-request-id   : 98fb5acd-7c9d-4bbc-aa01-f1b53ba1d698
x-ms-request-id               : f17f68a2-074a-44af-804b-1deaeb8b1231
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T025940Z:98fb5acd-7c9d-4bbc-aa01-f1b53ba1d698
X-Content-Type-Options        : nosniff
Date                          : Fri, 22 Dec 2023 02:59:40 GMT

Body:
{
  "name": "186f6061-4269-4aaa-936d-523481f042b8",
  "status": "InProgress",
  "startTime": "2023-12-22T02:58:09.030451Z",
  "error": {
    "code": "NotLatestOperation",
    "message": "Cannot proceed with the operation. Either the operation has been preempted by another one, or the information needed by the operation failed to be saved (or hasn't been saved yet)."
  }
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/xxxxxxxx-f2ecd6a53c42/providers/Microsoft.ContainerService/locations/canadaeast/operations/186f6061-4269-4aaa-936d-523481f042b8?api-version=2016-03-30&t=638388106895216045&c=MIIHHjCCBgagAwIBAgITOgI9LlF5jGEEQe1CmgAEAj0uUTANBgkqhkiG9w0BAQsFADBEMRMwEQYKCZImiZPyLGQBGRYDR0JMMRMwEQYKCZImiZPyLGQBGRYDQU1FMRgwFgYDVQQDEw9BTUUgSU5GUkEgQ0EgMDEwHhcNMjMxMTAxMjE0OTQ5WhcNMjQxMDI2MjE0OTQ5WjBAMT4wPAYDVQQDEzVhc3luY29wZXJhdGlvbnNpZ25pbmdjZXJ0aWZpY2F0ZS5tYW5hZ2VtZW50LmF6dXJlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMKt_zl2yRFDxIUmqoEA-B7WoRkSqscgwk6CGqDbwEVJS3oTI4Gk-ukQ6El3GQJVthiuwZGw4jwvTDyVWC8mQH2Zni6X1ocTXrNNa6VFJnvzeEn-HpmPATuII8AWtdtUnBvGZGAwgspvH8iZZCiHDowHCWhjVsU6_ipCsmzQeH3oRJI4AwUS_nCVA7U9WPdgIyQp6v5QPzcYCO_zIbNW4y_Cc3phfPEQvPvtSKhuijt68F5WlUY6Ps23ksosYG5VTx7z6skGG9arA5UKUzbbEO8lxzda3OFPZdG7OHe8-VMXxJjtSanZSSbdLgYBRB9m9PGr6voKtML9W5rEukE-lEUCAwEAAaOCBAswggQHMCcGCSsGAQQBgjcVCgQaMBgwCgYIKwYBBQUHAwEwCgYIKwYBBQUHAwIwPQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIhpDjDYTVtHiE8Ys-hZvdFs6dEoFggvX2K4Py0SACAWQCAQowggHaBggrBgEFBQcBAQSCAcwwggHIMGYGCCsGAQUFBzAChlpodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpaW5mcmEvQ2VydHMvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmwxLmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MFYGCCsGAQUFBzAChkpodHRwOi8vY3JsMi5hbWUuZ2JsL2FpYS9CWTJQS0lJTlRDQTAxLkFNRS5HQkxfQU1FJTIwSU5GUkElMjBDQSUyMDAxKDQpLmNydDBWBggrBgEFBQcwAoZKaHR0cDovL2NybDMuYW1lLmdibC9haWEvQlkyUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMElORlJBJTIwQ0ElMjAwMSg0KS5jcnQwVgYIKwYBBQUHMAKGSmh0dHA6Ly9jcmw0LmFtZS5nYmwvYWlhL0JZMlBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3J0MB0GA1UdDgQWBBRYUmiD-EjeFFNZByiKICN9p1JofzAOBgNVHQ8BAf8EBAMCBaAwggE1BgNVHR8EggEsMIIBKDCCASSgggEgoIIBHIZCaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraWluZnJhL0NSTC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMS5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMi5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsMy5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JshjRodHRwOi8vY3JsNC5hbWUuZ2JsL2NybC9BTUUlMjBJTkZSQSUyMENBJTIwMDEoNCkuY3JsMBcGA1UdIAQQMA4wDAYKKwYBBAGCN3sBATAfBgNVHSMEGDAWgBTl2Ztn_PjsurvwwKidileIud8-YzAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAKRwStAdrsy7tgEfmtXPZXIwgdXOYwSx2Kqzlhq8b6Ptid4V3vhJvDV0WLS7d72KIMUwZiBnfPHhMmsdkLOSpNSZYC23g3QJSBWvoS86Rpkeru5bEyQmv1gucF_MyaFaUXUd4b7xnvCkv-BBNO8m1lvFAowB2odz9esggf82yeOVBx0yEK_NNOK4l36XvRjvIqemSLZOSi1wow_wn_ovhAEPpSSN7UTahPtz7J12vhP1LuZ4Rvn_hNx48ncw445nMBnMVmcuuakaKDC6_sAcs5lg9jq2yHSaIHRw4xcoS5XwvIykGPMUyAD-dXWFih_XFOKh7ea50hObFhghf3Dtvlo&s=Dgt2QCbawbcFWOt9zWt1hj_3DStmg5bsbaznD9MzuUaVCPHJfp7y4o1u6huWIC7u7oenmATkuVsyWV1yEtXrycGPL0dOj_af6O5FQGy27nUDDOcDA1Gxv9cUfUrslRkd6H2EF8DqyoEpjLM1iwwl5H1MGOQogJ8y5NM2gVol1N3O18cZ7IZXzi4ddKmbL37iahdQhjoIqgIofpezdIvvW_pbxk0S10Y6TTOT8-CVOrBHZ2qVss_1wY7qfSlWMUQPZGBrmu6BdPUTCoQ5mSr-yA4rzrhQkMSuqWEch6GqIBpf425hIWO9TY8SIte9Cjf5GK_AwYIE-B98GrI1tSIlsA&h=CrBmGd1b7XJZiJkAsLonXmsv5P0Rs_rQRKDragy6rjk

Headers:
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-ratelimit-remaining-subscription-reads: 11992
x-ms-correlation-request-id   : 6061d146-4285-4df9-b62c-73084f3954d8
x-ms-request-id               : 74b71047-2bdb-48aa-87bd-0ec312a33d38
Strict-Transport-Security     : max-age=31536000; includeSubDomains
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T030010Z:6061d146-4285-4df9-b62c-73084f3954d8
X-Content-Type-Options        : nosniff
Date                          : Fri, 22 Dec 2023 03:00:09 GMT

Body:
{
  "name": "186f6061-4269-4aaa-936d-523481f042b8",
  "status": "Succeeded",
  "startTime": "2023-12-22T02:58:09.030451Z",
  "endTime": "2023-12-22T02:59:59.8463664Z",
  "error": {
    "code": "NotLatestOperation",
    "message": "Cannot proceed with the operation. Either the operation has been preempted by another one, or the information needed by the operation failed to be saved (or hasn't been saved yet)."
  }
}

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/xxxxxxxx-f2ecd6a53c42/resourceGroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload?api-version=2023-04-01

Headers:
x-ms-client-request-id        : bbb27b96-9862-4446-8a4f-92820ea20457

Body:

DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-routing-request-id       : SOUTHEASTASIA:20231222T030011Z:20694f96-000b-4e63-a3e5-c3e21e9d134e
x-ms-ratelimit-remaining-subscription-reads: 11991
x-ms-correlation-request-id   : 20694f96-000b-4e63-a3e5-c3e21e9d134e
x-ms-request-id               : ba403ac2-d489-4847-8e05-d6903e1f7db6
Strict-Transport-Security     : max-age=31536000; includeSubDomains
X-Content-Type-Options        : nosniff
Date                          : Fri, 22 Dec 2023 03:00:10 GMT

Body:
{
  "id": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourcegroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload",
  "location": "canadaeast",
  "name": "testworkload",
  "type": "Microsoft.ContainerService/ManagedClusters",
  "properties": {
    "provisioningState": "Succeeded",
    "powerState": {
      "code": "Running"
    },
    "kubernetesVersion": "1.26.10",
    "currentKubernetesVersion": "1.26.10",
    "dnsPrefix": "testworkload-dns",
    "fqdn": "testworkload-dns-h3jojj3y.hcp.canadaeast.azmk8s.io",
    "azurePortalFQDN": "testworkload-dns-h3jojj3y.portal.hcp.canadaeast.azmk8s.io",
    "agentPoolProfiles": [
      {
        "name": "agentpool",
        "count": 2,
        "vmSize": "Standard_DS2_v2",
        "osDiskSizeGB": 128,
        "osDiskType": "Managed",
        "kubeletDiskType": "OS",
        "maxPods": 110,
        "type": "VirtualMachineScaleSets",
        "maxCount": 5,
        "minCount": 2,
        "enableAutoScaling": true,
        "provisioningState": "Succeeded",
        "powerState": {
          "code": "Running"
        },
        "orchestratorVersion": "1.26.10",
        "currentOrchestratorVersion": "1.26.10",
        "enableNodePublicIP": false,
        "mode": "System",
        "osType": "Linux",
        "osSKU": "Ubuntu",
        "nodeImageVersion": "AKSUbuntu-2204gen2containerd-202312.06.0",
        "enableFIPS": false
      }
    ],
    "servicePrincipalProfile": {
      "clientId": "msi"
    },
    "addonProfiles": {
      "azureKeyvaultSecretsProvider": {
        "enabled": false,
        "config": null
      },
      "azurepolicy": {
        "enabled": false,
        "config": null
      }
    },
    "nodeResourceGroup": "MC_test_testworkload_canadaeast",
    "enableRBAC": true,
    "supportPlan": "KubernetesOfficial",
    "networkProfile": {
      "networkPlugin": "kubenet",
      "networkPolicy": "calico",
      "loadBalancerSku": "Standard",
      "loadBalancerProfile": {
        "managedOutboundIPs": {
          "count": 1
        },
        "effectiveOutboundIPs": [
          {
            "id": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourceGroups/MC_test_testworkload_canadaeast/providers/Microsoft.Network/publicIPAddresses/3e0d16f0-d311-4b62-a726-3073ca973d5e"
          }
        ]
      },
      "podCidr": "10.244.0.0/16",
      "serviceCidr": "10.0.0.0/16",
      "dnsServiceIP": "10.0.0.10",
      "outboundType": "loadBalancer",
      "podCidrs": [
        "10.244.0.0/16"
      ],
      "serviceCidrs": [
        "10.0.0.0/16"
      ],
      "ipFamilies": [
        "IPv4"
      ]
    },
    "maxAgentPools": 100,
    "identityProfile": {
      "kubeletidentity": {
        "resourceId": "/subscriptions/xxxxxxxx-f2ecd6a53c42/resourcegroups/MC_test_testworkload_canadaeast/providers/Microsoft.ManagedIdentity/userAssignedIdentities/testworkload-agentpool",
        "clientId": "4f9fe12e-ed17-45b6-bf7e-5f519c7cd096",
        "objectId": "fdaf2874-e45d-4b35-a70c-17eba6bd3be0"
      }
    },
    "autoScalerProfile": {
      "balance-similar-node-groups": "false",
      "expander": "random",
      "max-empty-bulk-delete": "10",
      "max-graceful-termination-sec": "600",
      "max-node-provision-time": "15m",
      "max-total-unready-percentage": "45",
      "new-pod-scale-up-delay": "0s",
      "ok-total-unready-count": "3",
      "scale-down-delay-after-add": "10m",
      "scale-down-delay-after-delete": "10s",
      "scale-down-delay-after-failure": "3m",
      "scale-down-unneeded-time": "10m",
      "scale-down-unready-time": "20m",
      "scale-down-utilization-threshold": "0.5",
      "scan-interval": "10s",
      "skip-nodes-with-local-storage": "false",
      "skip-nodes-with-system-pods": "true"
    },
    "autoUpgradeProfile": {
      "upgradeChannel": "patch"
    },
    "disableLocalAccounts": false,
    "securityProfile": {
      "workloadIdentity": {
        "enabled": true
      }
    },
    "storageProfile": {
      "diskCSIDriver": {
        "enabled": true
      },
      "fileCSIDriver": {
        "enabled": true
      },
      "snapshotController": {
        "enabled": true
      }
    },
    "oidcIssuerProfile": {
      "enabled": true,
      "issuerURL": "https://canadaeast.oic.prod-aks.azure.com/xxxxxxxx/edd8d595-5ec2-4231-8e6c-8346cc21e9ab/"
    },
    "workloadAutoScalerProfile": {}
  },
  "identity": {
    "type": "SystemAssigned",
    "principalId": "ba473d26-b261-433b-90a7-8a56a842b646",
    "tenantId": "xxxxxxxx"
  },
  "sku": {
    "name": "Base",
    "tier": "Free"
  }
}

PowerState               : Microsoft.Azure.Commands.Aks.Models.PSPowerState
Sku                      : Microsoft.Azure.Management.ContainerService.Models.ManagedClusterSKU
ExtendedLocation         : 
Identity                 : Microsoft.Azure.Commands.Aks.Models.PSManagedClusterIdentity
ProvisioningState        : Succeeded
MaxAgentPools            : 100
KubernetesVersion        : 1.26.10
CurrentKubernetesVersion : 1.26.10
DnsPrefix                : testworkload-dns
FqdnSubdomain            : 
Fqdn                     : testworkload-dns-h3jojj3y.hcp.canadaeast.azmk8s.io
PrivateFQDN              : 
AzurePortalFQDN          : testworkload-dns-h3jojj3y.portal.hcp.canadaeast.azmk8s.io
AgentPoolProfiles        : {agentpool}
LinuxProfile             : 
WindowsProfile           : 
ServicePrincipalProfile  : Microsoft.Azure.Commands.Aks.Models.PSContainerServiceServicePrincipalProfile
AddonProfiles            : {[azureKeyvaultSecretsProvider, Microsoft.Azure.Commands.Aks.Models.PSManagedClusterAddonProfile], [azurepolicy, Microsoft.Azure.Commands.Aks.Models.PSManagedClusterAddonProfile]}
PodIdentityProfile       : 
OidcIssuerProfile        : Microsoft.Azure.Management.ContainerService.Models.ManagedClusterOidcIssuerProfile
NodeResourceGroup        : MC_test_testworkload_canadaeast
EnableRBAC               : True
EnablePodSecurityPolicy  : 
NetworkProfile           : Microsoft.Azure.Commands.Aks.Models.PSContainerServiceNetworkProfile
AadProfile               : 
AutoUpgradeProfile       : Microsoft.Azure.Commands.Aks.Models.PSManagedClusterAutoUpgradeProfile
AutoScalerProfile        : Microsoft.Azure.Management.ContainerService.Models.ManagedClusterPropertiesAutoScalerProfile
ApiServerAccessProfile   : 
DiskEncryptionSetID      : 
IdentityProfile          : {[kubeletidentity, Microsoft.Azure.Commands.Aks.Models.PSManagedClusterPropertiesIdentityProfile]}
PrivateLinkResources     : 
DisableLocalAccounts     : False
HttpProxyConfig          : 
SecurityProfile          : Microsoft.Azure.Management.ContainerService.Models.ManagedClusterSecurityProfile
StorageProfile           : Microsoft.Azure.Management.ContainerService.Models.ManagedClusterStorageProfile
PublicNetworkAccess      : 
ResourceGroupName        : test
Id                       : /subscriptions/xxxxxxxx-f2ecd6a53c42/resourcegroups/test/providers/Microsoft.ContainerService/managedClusters/testworkload
Name                     : testworkload
Type                     : Microsoft.ContainerService/ManagedClusters
Location                 : canadaeast
Tags                     : 

DEBUG: 3:00:11 AM - [ConfigManager] Got nothing from [DisplayRegionIdentified], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 3:00:11 AM - [ConfigManager] Got nothing from [CheckForUpgrade], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: AzureQoSEvent:  Module: Az.Aks:6.0.0; CommandName: Set-AzAksCluster; PSVersion: 7.4.0; IsSuccess: True; Duration: 00:02:45.3406649
DEBUG: 3:00:11 AM - [ConfigManager] Got [True] from [EnableDataCollection], Module = [], Cmdlet = [].
DEBUG: 3:00:11 AM - SetAzureRmAks end processing.

Environment data

PS /home/max>> $PSVersionTable            

Name                           Value
----                           -----
PSVersion                      7.4.0
PSEdition                      Core
GitCommitId                    7.4.0
OS                             CBL-Mariner/Linux
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Module versions

PS /home/max>> Get-Module Az*

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Script     2.13.2                Az.Accounts                         {Add-AzEnvironment, Clear-AzConfig, Clear-AzContext, Clear-AzDefault…}
Script     6.0.0                 Az.Aks                              {Disable-AzAksAddOn, Enable-AzAksAddOn, Get-AzAksCluster, Get-AzAksNodePool…}
Script     7.1.0                 Az.Compute                          {Add-AzImageDataDisk, Add-AzVhd, Add-AzVMAdditionalUnattendContent, Add-AzVMDataDisk…}
Script     7.1.0                 Az.Network                          {Add-AzApplicationGatewayAuthenticationCertificate, Add-AzApplicationGatewayBackendAddressPool, Add-AzApplicationGatewayBackendHttpSetting, Add-AzApplicationGat…
Script     6.12.1                Az.Resources                        {Export-AzResourceGroup, Export-AzTemplateSpec, Get-AzDenyAssignment, Get-AzDeployment…}
Script     6.0.1                 Az.Storage                          {Add-AzRmStorageContainerLegalHold, Add-AzStorageAccountManagementPolicyAction, Add-AzStorageAccountNetworkRule, Close-AzStorageFileHandle…}
Script     1.1.3                 Az.Tools.Predictor                  {Disable-AzPredictor, Enable-AzPredictor, Open-AzPredictorSurvey, Send-AzPredictorRating}
Script     0.0.0.10              AzureAD.Standard.Preview            {Add-AzureADApplicationOwner, Add-AzureADDeviceRegisteredOwner, Add-AzureADDeviceRegisteredUser, Add-AzureADDirectoryRoleMember…}
Script     0.9.3                 AzurePSDrive

Error output

PS /home/max>> Resolve-AzError
DEBUG: 3:05:08 AM - ResolveError begin processing with ParameterSet 'AnyErrorParameterSet'.
DEBUG: 3:05:08 AM - using account id 'MSI@50342'...
DEBUG: 3:05:08 AM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarning], Module = [], Cmdlet = []. Returning default value [True].

DEBUG: 3:05:08 AM - [ConfigManager] Got nothing from [DisplayRegionIdentified], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 3:05:08 AM - [ConfigManager] Got nothing from [CheckForUpgrade], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: AzureQoSEvent:  Module: Az.Accounts:2.13.2; CommandName: Resolve-AzError; PSVersion: 7.4.0; IsSuccess: True; Duration: 00:00:00.0147060
DEBUG: 3:05:08 AM - [ConfigManager] Got [True] from [EnableDataCollection], Module = [], Cmdlet = [].
DEBUG: 3:05:08 AM - ResolveError end processing.
gsxmax commented 11 months ago

I beleive it is related to this code https://github.com/Azure/azure-powershell/blob/main/src/Aks/Aks/Commands/SetAzureRmAks.cs#L389, it would only check SP and work on AKS clusters using SP.

isra-fel commented 11 months ago

Thanks for reporting. @Nickcandy can you check with @wyunchi-ms and propose a fix? Thanks.