search

Azure / AKS

Azure Kubernetes Service
https://azure.github.io/AKS/
1.92k stars 289 forks source link

latency issue with Microsoft Entra groups when attaching ACR to AKS cluster using Entra group #4275

Closed krishna95sai closed 1 week ago

krishna95sai commented 3 weeks ago

Issue: latency issue with Microsoft Entra groups when attaching ACR to AKS cluster using Entra group

Added the AKS kubelet identity as a member of entra group ID Provided the ACR pull permission on the Entra group ID and tried to pull the image to AKS cluster and received the authentication error

Navigated to Access control IAM --> Check access and checked the access for kubelet identity and able to see the role for it under the ACR and still unable to pull the images to AKS cluster

Tried granting the ACR pull permission on ACR directly to kubelet identity and able to pull the images immediately(Removed it again)

Whereas observed the issue when added the kubelet identity as member of entra group ID

Referred the below document https://learn.microsoft.com/en-us/azure/aks/cluster-container-registry-integration?tabs=azure-cli

image

"There's a latency issue with Microsoft Entra groups when attaching ACR. If the AcrPull role is granted to a Microsoft Entra group and the kubelet identity is added to the group to complete the RBAC configuration, there may be a delay before the RBAC group takes effect. If you're running automation that requires the RBAC configuration to be complete, we recommend you use Bring your own kubelet identity as a workaround. You can pre-create a user-assigned identity, add it to the Microsoft Entra group, then use the identity as the kubelet identity to create an AKS cluster. This ensures the identity is added to the Microsoft Entra group before a token is generated by kubelet, which avoids the latency issue."

Tried to pull the images to AKS cluster again after couple of hours and still observed the issue When tried after around 24 hours it worked, able to pull the images in the AKS cluster (where kubelet identity is added as a member in entra ID and it has pull permission)

Ask: Wanted to know what is the approx latency time if we add the kubelet identity as member in entra ID and grant it ACR pull permission on ACR

krishna95sai commented 1 week ago

Found the document regarding the issue. It is a known limitation

https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/managed-identity-best-practice-recommendations#limitation-of-using-managed-identities-for-authorization