search

Azure / AKS

Azure Kubernetes Service
https://azure.github.io/AKS/
1.97k stars 307 forks source link

[BUG] AKS does not query MS Graph for group memberships when using Azure AD authentication with Kubernetes RBAC #3798

Open oddsund opened 1 year ago

oddsund commented 1 year ago

Not sure if its a bug or wrong documentation.

Describe the bug When updating one of our clusters to use Azure AD authentication with Kubernetes RBAC, the observed behaviour is that the JWT is verified, but no call is done to the MS Graph for group memberships.

According to the following docs; https://learn.microsoft.com/en-us/azure/aks/concepts-identity#azure-ad-integration The JWT should be verified, and then Webhook Token Authentication should be performed to assert current group memberships.

Our specific scenario is using PIM for Groups to elevate access temporarily.

This worked when we were using Azure AD authentication with Azure RBAC, albeit with up to 5 minute delay due to cache TTL and PIM for Azure Role. After the switch however, we have to acquire a new access token with an updated groups claim for the change to be reflected when we call the AKS cluster.

This also means that any group membership is retained for the duration of the token, so a revoked group membership is delayed by 90 minutes in the worst case.

Checking the kubelet config on the nodes also indicates that this should work. The flags --authentication-token-webhook=true --authorization-mode=Webhook --kubeconfig=/var/lib/kubelet/kubeconfig are set in /etc/default/kubelet, and the kubeconfig follows the spec as documented here: https://kubernetes.io/docs/reference/access-authn-authz/webhook/

To Reproduce

  1. Get credentials for AKS: az aks get-credentials -g <resourcegroup> -n <clustername>
  2. Run kubectl get pods to verify access
  3. Run kubectl get secret supersecret to verify that access is denied
  4. Add user to a group granting access to supersecret via (Cluster)RoleBinding
  5. After 5 minutes, run kubectl get secret supersecret. Access is still not granted
  6. Run kubelogin remove-tokens
  7. Run kubectl get secret supersecret again, perform any authentication necessary, and observe that access is granted

Expected behavior Access should be granted already in step 5, just like when using Azure AD authentication with Azure RBAC.

Environment (please complete the following information):

microsoft-github-policy-service[bot] commented 9 months ago

Action required from @Azure/aks-pm

microsoft-github-policy-service[bot] commented 8 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 8 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 7 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 7 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 6 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 6 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 5 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 5 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 4 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 4 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 3 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 3 months ago

Issue needing attention of @Azure/aks-leads

microsoft-github-policy-service[bot] commented 2 months ago

Issue needing attention of @Azure/aks-leads

karataliu commented 2 months ago

@oddsund Thanks for reporting this.

AAD supports 200 groups claim in JWT, refer to doc.

If current user has <= 200 groups, the client token will already contain groups claim. Server side WILL NOT perform additional call to fetch the groups. This helps reducing unnecessary external calls. If current user has > 200 groups, the client token doesn't contain group claim but a claim source URL. Server side will perform additional call to this URL to fetch groups.

For your case, it matches the behavior when user has <= 200 groups, such that server side will use the built-in group token. It is expected behavior to update kubeconfig credential to get the JIT applied.

oddsund commented 2 months ago

Thanks for the response @karataliu.

That's fine, but then this doc should be updated to clarify that; https://learn.microsoft.com/en-us/azure/aks/concepts-identity#azure-ad-integration

The docs currently states The server application uses user-provided credentials to query group memberships of the logged-in user from the MS Graph API., which is only true for user with > 200 groups.