azure aks and acr - failed: error getting credentials (context deadline exceeded)

Actual behavior

When launching the test.yaml I am met after ~20seconds with this error...

error checking push permissions -- make sure you entered the 
correct tag name, and that you are authenticated correctly, and try again: 
checking push permission for "MYACR.azurecr.io/mytestimage:test": 
resolving authorization for MYACR.azurecr.io failed: error getting credentials - 
err: exit status 1, out: `failed to acquire refresh token error refreshing sp token - adal: 
Failed to execute the refresh request. Error = 'context deadline exceeded'`

There is nothing else in the Pod logs.

Expected behavior Using MSI method, and not Service Principal, I am expecting the kaniko Pod to reach out successfully to the ACR. Following these instructions: https://github.com/GoogleContainerTools/kaniko#pushing-to-azure-container-registry without Service Principal.

To Reproduce Steps to reproduce the behavior:

Create an Azure ACR in same Azure Subscription
Create an AKS Cluster
Add roleassignment to allow AKS to access the ACR (Added "AcrPush" to agentpool using the kubelet identity ID).
Check the AKS can access ACR

az aks check-acr --acr MYACR.azurecr.io --name gdo-dev --resource-group gdo-dev-aks
Merged "gdo-dev" as current context in /tmp/tmpax9d6l7d
[2023-08-22T23:22:54Z] Checking host name resolution (MYACR.azurecr.io): SUCCEEDED
[2023-08-22T23:22:54Z] Canonical name for ACR (MYACR.azurecr.io): r0726eus.eastus.cloudapp.azure.com.
[2023-08-22T23:22:54Z] ACR location: eastus
[2023-08-22T23:22:54Z] Checking managed identity...
[2023-08-22T23:22:54Z] Kubelet managed identity client ID: 9a169892-a7e1-4535-abaf-a6edd7120999
[2023-08-22T23:22:54Z] Validating managed identity existance: SUCCEEDED
[2023-08-22T23:22:54Z] Validating image pull permission: SUCCEEDED
[2023-08-22T23:22:54Z] 
Your cluster can pull images from MYACR.azurecr.io!

Follow guide in kaniko to setup ACR registry
Use latest kaniko image.
Deploy a custom Pod that is an NGINX docker container

Additional Information

Dockerfile (very simple)

FROM nginx:latest
COPY ./index.html /usr/share/nginx/html/index.html

Build Context Dockerfile and index.html file as a .tar.gz file in an Azure Storage Blob.
Kaniko Image (fully qualified with digest) Latest.

Configured kaniko config.json to use ACR env but I am trying to use identityToken or username:password in .docker/acr/config.json. I have read somewhere that "acr-env" is ok, even if you're not using a Service Principal ENV.

{
  "credHelpers": {
    "MYACR.azurecr.io": "acr-env"
  }
}

Secret...

{
  "auths": {
        "MYACR.azurecr.io": {
        "auth":"<base64 username:password>"
      }

  }

Deployment test...

apiVersion: v1
kind: Pod
metadata:
  name: kaniko-mytestimage-test
  namespace: kaniko
spec:
  containers:
  - name: kaniko-test
    image: gcr.io/kaniko-project/executor:latest
    args:
    - "--dockerfile=Dockerfile"
    - "--context=https://MYBLOB.blob.core.windows.net/kaniko/kaniko-build-context.mytestimage.test.tar.gz"
    - "--destination=MYACR.azurecr.io/mytestimage:test"
    env:
    - name: AZURE_STORAGE_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: kaniko-build-secrets
          key: primary_access_key
    volumeMounts:
    - name: docker-config
      mountPath: /kaniko/.docker/
  volumes:
  - name: docker-config
    configMap:
      name: docker-config
  restartPolicy: Never

GoogleContainerTools / kaniko

azure aks and acr - failed: error getting credentials (context deadline exceeded) #2696