Support RoleAssignments to Kubernetes Namespaces (managedClusters/<>/namespaces/<ns>)

[kdandersen]

Hi all, hope it is OK to ask a question here.

I have searched “half the Internet”, the ASO2 documentation and also tried to see if a ASO2 Slack channel exists – with no avail.

I am trying to use ASO2 in a use case to deliver AKS Namespaces incl RBAC as a combination Kubernetes managed objects/CRDs such as ‘UserAssignedIdentity’ and ‘FederatedIdentityCredential’ objects for Namespace Service Account federated identities. And this works fine.

But I am also trying to use the ‘RoleAssignment’ CRD to grant access for users/groups at a Kubernetes Namespace scope, and I can’t see if this is at all possible.

Note, the AKS cluster is not a ASO2 managed/created object.

I would like to create the equivalent to the following AZ CLI configuration:

az role assignment create --role "Azure Kubernetes Service RBAC Writer" --assignee <principal_object_id> \
 --scope /subscriptions/<subscriptio_id>/resourceGroups/<resource_group>/providers/Microsoft.ContainerService/managedClusters/<aks_name>/namespaces/<namespace_name>

I have been able to make the ‘RoleAssignment’ work on a scope that is a ASO2 “managed” resource (Resource Group). See examples code below.

But is it at all possible to reference a ‘scope’ that isn’t known to or managed by ASO2 CRDs, as can be seen in the AZ CLI example?

EDIT: I actually managed to make this work by getting a hint from issue 3645 https://github.com/Azure/azure-service-operator/issues/3645

By making the

  owner:
    armId: /subscriptions/<subscriptio_id>/resourceGroups/<resource_group>/providers/Microsoft.ContainerService/managedClusters/<aks_name>/namespaces/<namespace_name>

This created the role assignment at of the 'Azure Kubernetes Service RBAC Writer' at the namespace scope of a referenced AKS cluster.

Kind regards Kim

apiVersion: resources.azure.com/v1api20200601
kind: ResourceGroup
metadata:
  name: <rg_name>
  namespace: <aso_namespace>
  annotations: 
    serviceoperator.azure.com/reconcile-policy: skip
spec:
  location: <az_region>
---
apiVersion: authorization.azure.com/v1api20220401
kind: RoleAssignment
metadata:
  name: aso2-demo-ra
  namespace: <aso_namespace>
spec:
  # This resource can be owner by any resource. In this example we've chosen a resource group for simplicity
  owner:
    name: <rg_name>
    group: resources.azure.com
    kind: ResourceGroup
  # This is the Principal ID of the AAD identity to which the role will be assigned
  # Example Azure EntraID Group
  principalId: <group_object_id>
  principalType: Group
  roleDefinitionReference:
    # 4d97b98b-1d4f-4787-a291-c67834d212e7 represents 'Network Contributor'
    armId: /subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/4d97b98b-1d4f-4787-a291-c67834d212e7

[theunrepentantgeek]

hope it is OK to ask a question here.

Absolutely!

tried to see if a ASO2 Slack channel exists

We have an ASO Slack channel over in the Kubernetes Slack here.

Note to self: make this easier to find

It looks as though you found one solution to your problem.

Another way would be to create the AKS cluster as an unmanaged ASO resource - add a resource for the cluster that includes the [annotation[(https://azure.github.io/azure-service-operator/guide/annotations/) serviceoperator.azure.com/reconcile-policy: skip. ASO will then know about the cluster (and be able to link in with other things), but won't do anything with it.

(Aside: I'm on leave until mid-January, so responses will be delayed.)

[kdandersen]

Hi Bevan,

Thanks for getting back to me.

Yes, the full 'armId' reference works as scope for the role assignment. Must say though that it wasn't immediately obvious that the 'owner:' object translated in to the assigned scope, but makes more sense when working a bit more with ASO2.

I did consider adding the AKS as a unmanaged resource via the reconcile-policy annotation. Did that for an already existing Resource Group when creating a UserAssignedIdentity. Don't know how how many attributes of an existing AKS are required to register this as an unmanaged resource...?

But the real issue is that the Kubernetes Namespace scoped role assignment extends the armId of the AKS cluster with .../namespaces/<namespace_name>.

And I assume this can only be achieved via the fully qualified resource id with the /namespaces/<namespace_name> appended.

All I really need to be able to fully close my use case would be: Feature: Azure ad groups https://github.com/Azure/azure-service-operator/issues/3541 So additional thumps up for that...

Enjoy your leave :-)

[matthchr]

All I really need to be able to fully close my use case would be: Feature: Azure ad groups https://github.com/Azure/azure-service-operator/issues/3541 So additional thumps up for that...

I think you want Kuberentes namespaces as a first class citizen, not Azure AD Groups? Azure AD groups are groups of users and are (AFAIK) unrelated to Kubernetes namespaces.

Can you expand a bit more on how you are using the identities assigned to this scope?

az role assignment create --role "Azure Kubernetes Service RBAC Writer" --assignee \ --scope /subscriptions//resourceGroups//providers/Microsoft.ContainerService/managedClusters//namespaces/

I'm trying to make sure I understand what you're trying to do. This seems to be granting the RBAC writer permissions at the scope of the Kubernetes namespace. I'm a bit confused because then your flow seems to be:

1. Run a pod as a particular service account.
2. Use that pods service account taken to exchange for an Azure AAD token (this is enabled by the FederatedIdentityCredential).
3. This AAD token can be taken to Azure and used to authenticate with the Kubernetes cluster the pod is running it (its APIServer). When used, it would have access only to the namespace specified.

What I don't quite follow is why you want AAD/Azure Identity involved here at all. If all you're wanting to do is control what access the service account has for the given namespace, why not just use the standard Kubernetes RBAC for service accounts?

Or is the point that the pod in cluster A is going to be accessing cluster B, so it's cross-cluster auth? If so, I definitely see the use, just want to make sure that's the goal here.

[kdandersen]

Hi Mathew Thanks for responding, and sorry for not being clear in my question. I have 2 uses cases I was working on:

UC 1. For an existing AKS (Terraform) + Namespace (ArgoCD) configure Azure Workload Identity to allow a Service Account to access Azure resources using a User Assigned Managed Identity

This UC I can solve using the AOS2 resources kind: UserAssignedIdentity and kind: FederatedIdentityCredential including some role assignments on the Azure resource.

UC 2. Access to AKS (Kubernetes) Namespaces for users/developers/Ops... via Entra ID (Azure AD) Groups and Role Assignments at a AKS Namespace Scope. https://learn.microsoft.com/en-us/azure/aks/manage-azure-rbac#create-role-assignments-for-users-to-access-the-cluster

In order to fully solve this UC in ASO2 I would need Entra ID Groups to be a ASO2 resource and also to have AKS Namespaces as a 'first class citizen'. But both these object types are managed by completely different APIs compared to the AzureRM that ASO2 uses.

In my Edit from the original post, I actually did manage to create the role assignment of a Entra ID group to a AKS Namespace using the ASO2 kind: RoleAssignment resource, but I did have to use the armId: of the AKS cluster with /namespaces/<ns_name> appended, create the Entra ID group manually and use the group principalId: together with the Azure role armId: for the roleDefinitionReference:.

This is what the embedded ASO2 yaml example in my original post shows. I am assigning an Entra ID group a role on a AKS namspace - not the user assigned managed identity.

Hope this explains. :-)

Thus my feedback that having ASO2 being able to create Groups in Entra ID would be nice...

[matthchr]

Thank you for that clarification - I missed it was two distinct use-cases. This helps clarify and now I understand the need for Entra group support as well!

[matthchr]

So really there are two asks here:

1. Kubernetes namespace support for role assignment (that's primarily what you can do with the managedClusters/<>/namespaces/<ns> resource.
2. Entra group support

[kdandersen]

Hi Matthew,

Yes - I guess that is basically what it boils down to.

But for now we have shifted our strategy and are creating RBAC via Terrform and the Namespaces will be managed using ArgoCD.

So no imminent needs from our side.

Don't know if you want to mark this request as closed? It would be all fine with me.

Rgds and thanks for the help...! Kim

[matthchr]

Given that there are two distinct asks here, and #2474 is already tracking one of them, I've gone ahead and changed the title of this issue to more accurately reflect the ask.