Closed tomkerkhove closed 2 years ago
Here are the checks I would do to investigate this issue (following the docs http://docs.promitor.io/walkthrough/use-promitor-with-managed-identity)
Be sure you have installed your AKS cluster using the managed identity option. Query the identity from the cluster:
echo "Retrieving cluster identity ID"
export aks_mi_identity="$(az aks show -g ${RG_NAME} -n ${CLUSTER_NAME} --query identityProfile.kubeletidentity.clientId -otsv)"
If I remember well, you may have the managed identity item also in the AKS resource group (called MC_....)
If you have created another managed identity (to respect the separation of concerns), be sure this identity has the "Monitor Reader" role assigned on your subscription / resource group
az role assignment list --assignee $AD_POD_IDENTITY_CLIENT_ID -g $RG_NAME | jq -r '.[].roleDefinitionName'
If you are using the system assigned identity, you can make the same check
From your cluster, check that your AAD Pod identity is correctly configured
kubectl get azureidentity
kubectl get azureidentitybinding
If you have correctly bind the AAD Pod Identity to your Managed Identity (System assigne or User Assigned) you should be able to use it. As an example, you can spin up a Azure CLI using this identity:
kubectl run azure-cli -it --image=mcr.microsoft.com/azure-cli --labels=aadpodidbinding=$AD_POD_IDENTITY_NAME /bin/bash
az login -i --debug
If you CLI is launching correctly that means your AAD pod Identity is configured correctly and ready to use.
Once you are sure everything is correctly configured on your AKS cluster, you can configure Promitor
azureAuthentication:
mode: SystemAssignedManagedIdentity
identity:
binding: <aad-pod-identity-name>
@Mimetis Thank you so much for clear instructions.
I think the issue was with authentication mode. Based on the steps mentioned in your comment I tried the command "az login -i --debug" and found that the identity which I was using SystemAssignedManagedIdentity and not UserAssignedManagedIdentity. The application started scraping metrics from Azure monitor without any issues after changing the azureAuthentication mode to "SystemAssignedManagedIdentity".
Thanks for your help.
Great to hear that helped. Would it help if this is a troubleshooting guide on docs.promitor.io @anoop2503?
Thank you @Mimetis!
Discussed in https://github.com/tomkerkhove/promitor/discussions/2011