[BUG] `keyvault-nginx-default` is stuck in ContainerCreating

[JoeyC-Dev]

Problem: Cannot set default certificate via Key Vault.

Screenshot:

Caption: I am not sure if I configured permission correctly, but the naming of "placeholder" and the use of "pause" image makes me thinking if this is intended. There is no other log I can find.

Set-up demo:

ranNum=$(echo $RANDOM)
region=westus
rG=aks_approuting_${ranNum}
kv=kv${ranNum}
aks=aks-${ranNum}
aksVer=1.30

cert_name=example-meow-${ranNum}

az group create -n ${rG} -l ${region} -o none

az aks create -n ${aks} -g ${rG} --kubernetes-version ${aksVer} --node-os-upgrade-channel None \
--node-vm-size Standard_A4_v2 --node-count 1 --enable-app-routing --no-ssh-key
infra_rG=$(az aks show -n ${aks} -g ${rG} --query nodeResourceGroup -o tsv)

# Section: set-policy mode
az keyvault create -n ${kv} -g ${rG} --enable-rbac-authorization false
az aks approuting update -n ${aks} -g ${rG} --enable-kv

# Grant permission
kvprovider_mi_client_id=$(az identity show --resource-group ${infra_rG} --name "azurekeyvaultsecretsprovider-${aks}" --query clientId -o tsv)
az keyvault set-policy -n ${kv} --certificate-permissions get --spn ${kvprovider_mi_client_id}

webapp_mi_client_id=$(az identity show --resource-group ${infra_rG} --name "webapprouting-${aks}" --query clientId -o tsv)
az keyvault set-policy -n ${kv} --certificate-permissions get --spn ${webapp_mi_client_id}

# Generate certificate
openssl req -new -x509 -nodes -subj "/CN=${cert_name}" -addext "subjectAltName=DNS:${cert_name}" -out ${cert_name}.crt -keyout ${cert_name}.key
openssl pkcs12 -export -in ${cert_name}.crt -inkey ${cert_name}.key -out ${cert_name}.pfx

# Import
az keyvault certificate import --vault-name ${kv} -n ${cert_name} -f ${cert_name}.pfx
certUrl=$(az keyvault certificate show --vault-name ${kv} -n ${cert_name} --query id -o tsv | sed -E 's/((.*)([\/]))([a-z0-9]+)/\2/')

# Get AKS credentials 
az aks get-credentials -n ${aks} -g ${rG}

# Apply default cert
cat <<EOF | kubectl apply -f -
apiVersion: approuting.kubernetes.azure.com/v1alpha1
kind: NginxIngressController
metadata:
  name: default
spec:
  ingressClassName: webapprouting.kubernetes.azure.com
  controllerNamePrefix: nginx
  loadBalancerAnnotations: 
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
  defaultSSLCertificate:
    keyVaultURI: ${certUrl}
EOF

[JoeyC-Dev]

Tried:

az aks approuting update -n ${aks} -g ${rG} --enable-kv --attach-kv ${kvURI}

And this work.

I want to know why I have to attach-kv here? I should already give enough permission.

[sabbour]

The az keyvault set-policy -n ${kv} --certificate-permissions get --spn ${webapp_mi_client_id} command should be az keyvault set-policy -n ${kv} --certificate-permissions get --object-id ${webapp_mi_client_id} because it is a managed identity and not a service principal.

attach-kv does this for you.