search

Azure / azure-cli-extensions

Public Repository for Extensions of Azure CLI.
https://docs.microsoft.com/en-us/cli/azure
MIT License
382 stars 1.21k forks source link

Error deploying App Service Extension for Arc Kubernetes #3661

Open ghost opened 3 years ago

ghost commented 3 years ago

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name az customlocation create Extension Name: customlocation. Version: 0.1.2.

Errors:

FailedToLoadClusterExtensionConfig: Combined Getting Extension Config Errors: ,
 Error for extension ID /subscriptions/d40ad3b0-3405-4c2e-b618-87ea80548513/resourceGroups/DefaultResourceGroup-WEU/providers/Microsoft.Kubernetes/connectedClusters/WestEuropeCluster/providers/Microsoft.KubernetesConfiguration/extensions/AppServiceExt : Invalid InstallState of Cluster Extension. InstallState is : Failed

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Expected Behavior

Environment Summary

Windows-10-10.0.19041-SP0
Python 3.8.9
Installer: MSI

azure-cli 2.26.1

Extensions:
appservice-kube 0.1.20
connectedk8s 1.1.5
customlocation 0.1.2
k8s-extension 0.5.1

Additional Context

ghost commented 3 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @antcp, @AzureAppServiceCLI.

Issue Details
### **This is autogenerated. Please review and update as needed.** ## Describe the bug **Command Name** `az customlocation create Extension Name: customlocation. Version: 0.1.2.` **Errors:** ``` FailedToLoadClusterExtensionConfig: Combined Getting Extension Config Errors: , Error for extension ID /subscriptions/d40ad3b0-3405-4c2e-b618-87ea80548513/resourceGroups/DefaultResourceGroup-WEU/providers/Microsoft.Kubernetes/connectedClusters/WestEuropeCluster/providers/Microsoft.KubernetesConfiguration/extensions/AppServiceExt : Invalid InstallState of Cluster Extension. InstallState is : Failed ``` ## To Reproduce: Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information. - _Put any pre-requisite steps here..._ - `az customlocation create -g {} -n {} --host-resource-id {} --namespace {} -c {} -l {} --query {} -o {}` ## Expected Behavior ## Environment Summary ``` Windows-10-10.0.19041-SP0 Python 3.8.9 Installer: MSI azure-cli 2.26.1 Extensions: appservice-kube 0.1.20 connectedk8s 1.1.5 customlocation 0.1.2 k8s-extension 0.5.1 ``` ## Additional Context
Author: AlexEnfoMMX
Assignees: -
Labels: `App Services`, `Service Attention`
Milestone: -
yonzhan commented 3 years ago

route to service team

markjgardner commented 3 years ago

I'm running into this error as well. Digging a little deeper I'm seeing the following error on the extension, looks like the helm install times out waiting on some condition:

> az k8s-extension show -g "myresourcegroup" --name "lima-test" --cluster-type connectedClusters -c "testcluster"

...
  "errorInfo": {
    "code": "InstallationFailed",
    "message": "Error: {failed to install chart from path [] for release [lima-test]: err [release lima-test failed, and has been uninstalled due to atomic being set: timed out waiting for the condition]} occurred while doing the operation : {Installing the extension} on the config"
  },
...

It looks like all of the components installed successfully even though the extension is in a failed state.

$ k get all -n <my-namespace>

NAME                                                            READY   STATUS    RESTARTS   AGE
pod/lima-test-k8se-activator-5c66bc5954-2l674                1/1     Running   0          2m28s
pod/lima-test-k8se-app-controller-7657fdb57b-9ncvx           1/1     Running   0          2m28s
pod/lima-test-k8se-build-service-6f445fd7cc-gtlk5            1/1     Running   0          2m28s
pod/lima-test-k8se-envoy-7bd6489cdb-7rqpq                    1/1     Running   0          2m27s
pod/lima-test-k8se-envoy-7bd6489cdb-8nxf7                    1/1     Running   0          2m27s
pod/lima-test-k8se-envoy-7bd6489cdb-td6xz                    1/1     Running   0          2m27s
pod/lima-test-k8se-http-scaler-65ff596b9f-nr2lf              1/1     Running   0          2m28s
pod/lima-test-k8se-img-cacher-x58j5                          1/1     Running   0          2m28s
pod/lima-test-k8se-img-cacher-xjv5t                          1/1     Running   0          2m28s
pod/lima-test-k8se-keda-metrics-apiserver-7d58559ff4-7czhl   1/1     Running   0          2m27s
pod/lima-test-k8se-keda-operator-67bf697554-tsx7x            1/1     Running   0          2m28s
pod/lima-test-k8se-log-processor-kbkbc                       1/1     Running   3          2m28s
pod/lima-test-k8se-log-processor-vkpdk                       1/1     Running   0          2m28s
pod/lima-test-k8se-log-processor-wrdgh                       1/1     Running   2          2m28s

NAME                                               TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                                     AGE
service/lima-test-k8se-activator                ClusterIP      10.0.216.87    <none>        4045/TCP,4050/TCP,4046/TCP                  2m29s
service/lima-test-k8se-app-controller           ClusterIP      10.0.180.2     <none>        9090/TCP                                    2m29s
service/lima-test-k8se-build-service            ClusterIP      10.0.122.58    <none>        8181/TCP                                    2m29s
service/lima-test-k8se-envoy                    LoadBalancer   10.0.191.197   <pending>     80:30772/TCP,443:31414/TCP,8081:31298/TCP   2m29s
service/lima-test-k8se-http-scaler              ClusterIP      10.0.132.69    <none>        4055/TCP,4050/TCP                           2m29s
service/lima-test-k8se-keda-metrics-apiserver   ClusterIP      10.0.65.4      <none>        443/TCP,80/TCP                              2m29s

NAME                                             DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/lima-test-k8se-img-cacher      2         2         2       2            2           <none>          2m29s
daemonset.apps/lima-test-k8se-log-processor   3         3         3       3            3           <none>          2m29s

NAME                                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/lima-test-k8se-activator                1/1     1            1           2m29s
deployment.apps/lima-test-k8se-app-controller           1/1     1            1           2m29s
deployment.apps/lima-test-k8se-build-service            1/1     1            1           2m29s
deployment.apps/lima-test-k8se-envoy                    3/3     3            3           2m29s
deployment.apps/lima-test-k8se-http-scaler              1/1     1            1           2m29s
deployment.apps/lima-test-k8se-keda-metrics-apiserver   1/1     1            1           2m29s
deployment.apps/lima-test-k8se-keda-operator            1/1     1            1           2m29s

NAME                                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/lima-test-k8se-activator-5c66bc5954                1         1         1       2m29s
replicaset.apps/lima-test-k8se-app-controller-7657fdb57b           1         1         1       2m29s
replicaset.apps/lima-test-k8se-build-service-6f445fd7cc            1         1         1       2m29s
replicaset.apps/lima-test-k8se-envoy-7bd6489cdb                    3         3         3       2m29s
replicaset.apps/lima-test-k8se-http-scaler-65ff596b9f              1         1         1       2m29s
replicaset.apps/lima-test-k8se-keda-metrics-apiserver-7d58559ff4   1         1         1       2m29s
replicaset.apps/lima-test-k8se-keda-operator-67bf697554            1         1         1       2m29s
jaromirk commented 2 years ago

I'm facing the same issue. Here is my end-to-end scenario https://github.com/microsoft/MSLab/blob/dev/Scenarios/AzSHCI%20and%20Kubernetes/Scenario.ps1

Any update please?

NAME                                                              READY   STATUS    RESTARTS   AGE
pod/appservice-ext-k8se-activator-5d4ffc9587-2zq8m                1/1     Running   0          4h31m
pod/appservice-ext-k8se-activator-5d4ffc9587-q5chm                1/1     Running   0          4h31m
pod/appservice-ext-k8se-app-controller-74bb978dcd-2q7g6           1/1     Running   0          4h31m
pod/appservice-ext-k8se-app-controller-74bb978dcd-trm2h           1/1     Running   0          4h31m
pod/appservice-ext-k8se-build-service-774cfcc8df-j7j6p            2/2     Running   0          4h31m
pod/appservice-ext-k8se-envoy-7d74bc8757-5kj4p                    1/1     Running   0          4h31m
pod/appservice-ext-k8se-envoy-7d74bc8757-6mwqz                    1/1     Running   0          4h31m
pod/appservice-ext-k8se-envoy-7d74bc8757-vtl7p                    1/1     Running   0          4h31m
pod/appservice-ext-k8se-envoy-controller-74bd9f6994-lcn9w         1/1     Running   0          4h31m
pod/appservice-ext-k8se-envoy-controller-74bd9f6994-vhj4w         1/1     Running   0          4h31m
pod/appservice-ext-k8se-http-scaler-6f9c6b8f4f-z2jkz              1/1     Running   0          4h31m
pod/appservice-ext-k8se-img-cacher-5jcw9                          1/1     Running   0          4h31m
pod/appservice-ext-k8se-keda-metrics-apiserver-7bd6798884-j4sxk   1/1     Running   0          4h31m
pod/appservice-ext-k8se-keda-operator-6b976878d9-fzkj9            1/1     Running   0          4h31m
pod/appservice-ext-k8se-log-processor-ql565                       1/1     Running   0          4h31m
pod/appservice-ext-k8se-log-processor-qqtfl                       1/1     Running   0          4h31m

NAME                                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/appservice-ext-k8se-activator                ClusterIP      10.99.138.161    <none>        4045/TCP,4050/TCP,4046/TCP   4h31m
service/appservice-ext-k8se-build-service            ClusterIP      10.111.116.201   <none>        8181/TCP,4343/TCP            4h31m
service/appservice-ext-k8se-envoy                    LoadBalancer   10.106.219.57    10.0.1.5      80:32212/TCP,443:30373/TCP   4h31m
service/appservice-ext-k8se-envoy-controller         ClusterIP      10.102.254.234   <none>        9090/TCP                     4h31m
service/appservice-ext-k8se-envoy-internal           ClusterIP      10.97.150.235    <none>        80/TCP,443/TCP               4h31m
service/appservice-ext-k8se-http-scaler              ClusterIP      10.97.178.142    <none>        4055/TCP,4050/TCP            4h31m
service/appservice-ext-k8se-keda-metrics-apiserver   ClusterIP      10.104.200.116   <none>        443/TCP,80/TCP               4h31m

NAME                                               DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/appservice-ext-k8se-img-cacher      1         1         1       1            1           <none>          4h31m
daemonset.apps/appservice-ext-k8se-log-processor   2         2         2       2            2           <none>          4h31m

NAME                                                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/appservice-ext-k8se-activator                2/2     2            2           4h31m
deployment.apps/appservice-ext-k8se-app-controller           2/2     2            2           4h31m
deployment.apps/appservice-ext-k8se-build-service            1/1     1            1           4h31m
deployment.apps/appservice-ext-k8se-envoy                    3/3     3            3           4h31m
deployment.apps/appservice-ext-k8se-envoy-controller         2/2     2            2           4h31m
deployment.apps/appservice-ext-k8se-http-scaler              1/1     1            1           4h31m
deployment.apps/appservice-ext-k8se-keda-metrics-apiserver   1/1     1            1           4h31m
deployment.apps/appservice-ext-k8se-keda-operator            1/1     1            1           4h31m

NAME                                                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/appservice-ext-k8se-activator-5d4ffc9587                2         2         2       4h31m
replicaset.apps/appservice-ext-k8se-app-controller-74bb978dcd           2         2         2       4h31m
replicaset.apps/appservice-ext-k8se-build-service-774cfcc8df            1         1         1       4h31m
replicaset.apps/appservice-ext-k8se-envoy-7d74bc8757                    3         3         3       4h31m
replicaset.apps/appservice-ext-k8se-envoy-controller-74bd9f6994         2         2         2       4h31m
replicaset.apps/appservice-ext-k8se-http-scaler-6f9c6b8f4f              1         1         1       4h31m
replicaset.apps/appservice-ext-k8se-keda-metrics-apiserver-7bd6798884   1         1         1       4h31m
replicaset.apps/appservice-ext-k8se-keda-operator-6b976878d9            1         1         1       4h31m

NAME                                                                     REFERENCE                              TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/keda-hpa-appservice-ext-k8se-envoy   Deployment/appservice-ext-k8se-envoy   <unknown>/50%   3         5         3          4h31m
panchagnula commented 2 years ago

@StrawnSC could you take a look, if this infact related to our extension? Thanks!

jaromirk commented 2 years ago

@StrawnSC feel free to ping me on teams via jaromir.kaspar@dell.com if anything... I have also issues deploying DC extension. It's in the same scenario. If it's also your extension, it would be awesome if we could review it.

StrawnSC commented 2 years ago

Hi, it looks like you're using the private preview version of the appservice-kube extension, which is no longer supported. Could you switch to the new public extension by running the following commands: 

az extension remove -n "appservice-kube"
az extension add -n "appservice-kube"
jaromirk commented 2 years ago

I'm pretty sure I was doing it. It was recently changed? https://github.com/microsoft/MSLab/blob/dev/Scenarios/AzSHCI%20and%20Kubernetes/Scenario.ps1#L890-L891

jaromirk commented 2 years ago

image

image

NAME READY STATUS RESTARTS AGE pod/appservice-ext-k8se-activator-5d4ffc9587-28jgl 1/1 Running 0 7m46s pod/appservice-ext-k8se-activator-5d4ffc9587-dzd5w 1/1 Running 0 7m46s pod/appservice-ext-k8se-app-controller-74bb978dcd-bvqq5 1/1 Running 0 7m46s pod/appservice-ext-k8se-app-controller-74bb978dcd-wbmb7 1/1 Running 0 7m45s pod/appservice-ext-k8se-build-service-5f9f676645-7ztch 2/2 Running 0 7m45s pod/appservice-ext-k8se-envoy-6b64f9f9ff-5lgv4 1/1 Running 0 7m45s pod/appservice-ext-k8se-envoy-6b64f9f9ff-np9bq 1/1 Running 0 7m45s pod/appservice-ext-k8se-envoy-6b64f9f9ff-pl4wp 1/1 Running 0 7m45s pod/appservice-ext-k8se-envoy-controller-9c7c47bfd-4ph8w 1/1 Running 0 7m45s pod/appservice-ext-k8se-envoy-controller-9c7c47bfd-rfr4c 1/1 Running 0 7m45s pod/appservice-ext-k8se-http-scaler-5b4486657b-v8m75 1/1 Running 0 7m45s pod/appservice-ext-k8se-img-cacher-sr5fx 1/1 Running 0 7m46s pod/appservice-ext-k8se-keda-metrics-apiserver-7bd6798884-fl7h5 1/1 Running 0 7m45s pod/appservice-ext-k8se-keda-operator-6b976878d9-xs4n9 1/1 Running 0 7m45s pod/appservice-ext-k8se-log-processor-54zzh 1/1 Running 0 7m46s pod/appservice-ext-k8se-log-processor-8svlw 1/1 Running 0 7m46s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/appservice-ext-k8se-activator ClusterIP 10.105.59.199 4045/TCP,4050/TCP,4046/TCP 7m46s service/appservice-ext-k8se-build-service ClusterIP 10.109.69.233 8181/TCP,4343/TCP 7m46s service/appservice-ext-k8se-envoy LoadBalancer 10.109.216.191 10.0.1.5 80:30505/TCP,443:32556/TCP 7m46s service/appservice-ext-k8se-envoy-controller ClusterIP 10.104.210.69 9090/TCP 7m46s service/appservice-ext-k8se-envoy-internal ClusterIP 10.100.231.84 80/TCP,443/TCP 7m46s service/appservice-ext-k8se-http-scaler ClusterIP 10.104.81.47 4055/TCP,4050/TCP 7m46s service/appservice-ext-k8se-keda-metrics-apiserver ClusterIP 10.111.63.119 443/TCP,80/TCP 7m46s

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/appservice-ext-k8se-img-cacher 1 1 1 1 1 7m46s daemonset.apps/appservice-ext-k8se-log-processor 2 2 2 2 2 7m46s

NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/appservice-ext-k8se-activator 2/2 2 2 7m46s deployment.apps/appservice-ext-k8se-app-controller 2/2 2 2 7m46s deployment.apps/appservice-ext-k8se-build-service 1/1 1 1 7m46s deployment.apps/appservice-ext-k8se-envoy 3/3 3 3 7m45s deployment.apps/appservice-ext-k8se-envoy-controller 2/2 2 2 7m45s deployment.apps/appservice-ext-k8se-http-scaler 1/1 1 1 7m45s deployment.apps/appservice-ext-k8se-keda-metrics-apiserver 1/1 1 1 7m45s deployment.apps/appservice-ext-k8se-keda-operator 1/1 1 1 7m45s

NAME DESIRED CURRENT READY AGE replicaset.apps/appservice-ext-k8se-activator-5d4ffc9587 2 2 2 7m46s replicaset.apps/appservice-ext-k8se-app-controller-74bb978dcd 2 2 2 7m46s replicaset.apps/appservice-ext-k8se-build-service-5f9f676645 1 1 1 7m46s replicaset.apps/appservice-ext-k8se-envoy-6b64f9f9ff 3 3 3 7m45s replicaset.apps/appservice-ext-k8se-envoy-controller-9c7c47bfd 2 2 2 7m45s replicaset.apps/appservice-ext-k8se-http-scaler-5b4486657b 1 1 1 7m45s replicaset.apps/appservice-ext-k8se-keda-metrics-apiserver-7bd6798884 1 1 1 7m45s replicaset.apps/appservice-ext-k8se-keda-operator-6b976878d9 1 1 1 7m45s

StrawnSC commented 2 years ago

@jaromirk sorry about that -- looks like you're right. I was going off of the environment for @AlexEnfoMMX, which lists the appservice-kube version as 0.1.20 instead of 0.1.2. This error seems to be caused by invoking the k8s-extension and customlocation cli extensions, so I'll try and loop in the code owners for those CLI extensions

apwestgarth commented 2 years ago
  1. @markjgardner looking at the output of kubectl get services -n I can see the envoy loadbalancer service is not being assigned an IP address hence why the extension is in a failed state: NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/lima-test-k8se-activator ClusterIP 10.0.216.87 4045/TCP,4050/TCP,4046/TCP 2m29s service/lima-test-k8se-app-controller ClusterIP 10.0.180.2 9090/TCP 2m29s service/lima-test-k8se-build-service ClusterIP 10.0.122.58 8181/TCP 2m29s service/lima-test-k8se-envoy LoadBalancer 10.0.191.197 80:30772/TCP,443:31414/TCP,8081:31298/TCP 2m29s service/lima-test-k8se-http-scaler ClusterIP 10.0.132.69 4055/TCP,4050/TCP 2m29s service/lima-test-k8se-keda-metrics-apiserver ClusterIP 10.0.65.4 443/TCP,80/TCP 2m29s

Is the service still in that state? You cannot associate a Custom Location with a failed extension install.

  1. @AlexEnfoMMX please share the output of both the az k8s-extension show command, providing the extension name, connectedCluster id etc as per documentation and the output of kubectl get services -n

  2. @jaromirk please share the output of both the az k8s-extension show command, providing the extension name, connectedCluster id etc as per documentation

jaromirk commented 2 years ago

It looks like extension was now deployed successfully. Let me wipe the environment and rebuild. Will let you know. (I had some issues creating environment - az appservice kube create)...

Btw. In scenario i created data services first (same cluster, different namespace). Is it good approach? I have one scenario, where I deploy one kubernetes cluster where I would like to demonstrate creating kubernetes cluster, enable monitoring, policies, Arc and all other Azure features. Then I would create on the same cluster (different NS) app services and data services. And data services in custom storage class (in Azure Stack HCI, custom container).

az k8s-extension show --resource-group $resourcegroup --cluster-name $KubernetesClusterName --cluster-type connectedClusters --name $extensionName | convertfrom-json

aksAssignedIdentity            :
autoUpgradeMinorVersion        : True
configurationProtectedSettings : @{logProcessor.appLogs.logAnalyticsConfig.customerId=;
                                 logProcessor.appLogs.logAnalyticsConfig.sharedKey=}
configurationSettings          : @{Microsoft.CustomLocation.ServiceAccount=default; appsNamespace=appservice-ns;
                                 buildService.storageAccessMode=ReadWriteOnce; buildService.storageClassName=default;
                                 clusterName=demo; customConfigMap=appservice-ns/kube-environment-config; envoy.annotat
                                 ions.service.beta.kubernetes.io/azure-load-balancer-resource-group=AksHCI-Cluster-rg;
                                 keda.enabled=true; logProcessor.appLogs.destination=log-analytics}
customLocationSettings         :
errorInfo                      :
extensionType                  : microsoft.web.appservice
id                             : /subscriptions/0d2cb74f-ef20-4daf-9888-32c8a4846fcf/resourceGroups/AksHCI-Cluster-rg/p
                                 roviders/Microsoft.Kubernetes/connectedClusters/demo/providers/Microsoft.KubernetesCon
                                 figuration/extensions/appservice-ext
identity                       : @{principalId=879d5369-36c0-4fb9-a819-123b4681d520; tenantId=; type=SystemAssigned}
location                       :
name                           : appservice-ext
packageUri                     :
provisioningState              : Succeeded
releaseTrain                   : stable
resourceGroup                  : AksHCI-Cluster-rg
scope                          : @{cluster=; namespace=}
statuses                       : {}
systemData                     : @{createdAt=2022-01-26T08:56:27.458101+00:00; createdBy=; createdByType=;
                                 lastModifiedAt=2022-01-26T08:56:27.458101+00:00; lastModifiedBy=; lastModifiedByType=}
type                           : Microsoft.KubernetesConfiguration/extensions
version                        : 0.11.1
jaromirk commented 2 years ago

New fresh deployment...

PS C:\Windows\system32>     az k8s-extension create `
>>     --resource-group $resourcegroup `
>>     --name $extensionName `
>>     --cluster-type connectedClusters `
>>     --cluster-name $KubernetesClusterName `
>>     --extension-type 'Microsoft.Web.Appservice' `
>>     --release-train stable `
>>     --auto-upgrade-minor-version true `
>>     --scope cluster `
>>     --release-namespace $namespace `
>>     --configuration-settings "Microsoft.CustomLocation.ServiceAccount=default" `
>>     --configuration-settings "appsNamespace=${namespace}" `
>>     --configuration-settings "clusterName=${kubeEnvironmentName}" `
>>     --configuration-settings "keda.enabled=true" `
>>     --configuration-settings "buildService.storageClassName=default" `
>>     --configuration-settings "buildService.storageAccessMode=ReadWriteOnce" `
>>     --configuration-settings "customConfigMap=${namespace}/kube-environment-config" `
>>     --configuration-settings "envoy.annotations.service.beta.kubernetes.io/azure-load-balancer-resource-group=${aksClusterGroupName}" `
>>     --configuration-settings "logProcessor.appLogs.destination=log-analytics" `
>>     --configuration-protected-settings "logProcessor.appLogs.logAnalyticsConfig.customerId=${logAnalyticsWorkspaceIdEnc}" `
>>     --configuration-protected-settings "logProcessor.appLogs.logAnalyticsConfig.sharedKey=${logAnalyticsKeyEnc}"
(ExtensionCreationFailed)  error: Unable to get the status from the local CRD with the error : {Error : Retry for given duration didn't get any results with err {status not populated}}
Code: ExtensionCreationFailed
Message:  error: Unable to get the status from the local CRD with the error : {Error : Retry for given duration didn't get any results with err {status not populated}}
PS C:\Windows\system32>
PS C:\Windows\system32>     az k8s-extension show --resource-group $resourcegroup --cluster-name $KubernetesClusterName --cluster-type connectedClusters --name $extensionName | convertfrom-json

aksAssignedIdentity            :
autoUpgradeMinorVersion        : True
configurationProtectedSettings : @{logProcessor.appLogs.logAnalyticsConfig.customerId=;
                                 logProcessor.appLogs.logAnalyticsConfig.sharedKey=}
configurationSettings          : @{Microsoft.CustomLocation.ServiceAccount=default; appsNamespace=appservice-ns;
                                 buildService.storageAccessMode=ReadWriteOnce; buildService.storageClassName=default;
                                 clusterName=demo; customConfigMap=appservice-ns/kube-environment-config; envoy.annotat
                                 ions.service.beta.kubernetes.io/azure-load-balancer-resource-group=AksHCI-Cluster-rg;
                                 keda.enabled=true; logProcessor.appLogs.destination=log-analytics}
customLocationSettings         :
errorInfo                      :
extensionType                  : microsoft.web.appservice
id                             : /subscriptions/0d2cb74f-ef20-4daf-9888-32c8a4846fcf/resourceGroups/AksHCI-Cluster-rg/p
                                 roviders/Microsoft.Kubernetes/connectedClusters/demo/providers/Microsoft.KubernetesCon
                                 figuration/extensions/appservice-ext
identity                       : @{principalId=7006a4d6-8dae-4377-9a46-5df02a631b92; tenantId=; type=SystemAssigned}
location                       :
name                           : appservice-ext
packageUri                     :
provisioningState              : Failed
releaseTrain                   : stable
resourceGroup                  : AksHCI-Cluster-rg
scope                          : @{cluster=; namespace=}
statuses                       : {@{code=InstallationFailed; displayStatus=; level=; message=error: Unable to get the
                                 status from the local CRD with the error : {Error : Retry for given duration didn't
                                 get any results with err {status not populated}}; time=}}
systemData                     : @{createdAt=2022-01-27T11:36:56.060851+00:00; createdBy=; createdByType=;
                                 lastModifiedAt=2022-01-27T11:36:56.060851+00:00; lastModifiedBy=; lastModifiedByType=}
type                           : Microsoft.KubernetesConfiguration/extensions
version                        : 0.11.1

PS C:\Windows\system32>
jaromirk commented 2 years ago

And after some time (like 5 minutes or so) it was successfully created :-O

az k8s-extension show --resource-group $resourcegroup --cluster-name $KubernetesClusterName --cluster-type connectedClusters --name $extensionName | convertfrom-json
aksAssignedIdentity            :
autoUpgradeMinorVersion        : True
configurationProtectedSettings : @{logProcessor.appLogs.logAnalyticsConfig.customerId=; logProcessor.appLogs.logAnalyticsConfig.sharedKey=}
configurationSettings          : @{Microsoft.CustomLocation.ServiceAccount=default; appsNamespace=appservice-ns; buildService.storageAccessMode=ReadWriteOnce;
                                 buildService.storageClassName=default; clusterName=demo; customConfigMap=appservice-ns/kube-environment-config;
                                 envoy.annotations.service.beta.kubernetes.io/azure-load-balancer-resource-group=AksHCI-Cluster-rg; keda.enabled=true;
                                 logProcessor.appLogs.destination=log-analytics}
customLocationSettings         :
errorInfo                      :
extensionType                  : microsoft.web.appservice
id                             : /subscriptions/0d2cb74f-ef20-4daf-9888-32c8a4846fcf/resourceGroups/AksHCI-Cluster-rg/providers/Microsoft.Kubernetes/connectedClusters/
                                 demo/providers/Microsoft.KubernetesConfiguration/extensions/appservice-ext
identity                       : @{principalId=7006a4d6-8dae-4377-9a46-5df02a631b92; tenantId=; type=SystemAssigned}
location                       :
name                           : appservice-ext
packageUri                     :
provisioningState              : Succeeded
releaseTrain                   : stable
resourceGroup                  : AksHCI-Cluster-rg
scope                          : @{cluster=; namespace=}
statuses                       : {}
systemData                     : @{createdAt=2022-01-27T11:36:56.060851+00:00; createdBy=; createdByType=; lastModifiedAt=2022-01-27T11:36:56.060851+00:00;
                                 lastModifiedBy=; lastModifiedByType=}
type                           : Microsoft.KubernetesConfiguration/extensions
version                        : 0.11.1
jaromirk commented 2 years ago

Tested it once more, I added script that will wait some time as the same "issue" occurred. It did not finish successfully (ended with provisioningState failed). Takes around 5 minutes... 

    do {
        Write-Host "." -NoNewLine
        Start-Sleep 10
        $Extension=az k8s-extension show --resource-group $resourcegroup --cluster-name $KubernetesClusterName --cluster-type connectedClusters --name $extensionName | convertfrom-json
    } until (
        $extension.provisioningstate -eq "Succeeded"
    )