search

kubernetes-sigs / azurefile-csi-driver

Azure File CSI Driver
Apache License 2.0
149 stars 136 forks source link

Connection to storage account with storage account keys disabled doesn't work with workload identity (kerberos auth support) #1737

Open Vegoo89 opened 4 months ago

Vegoo89 commented 4 months ago

What happened: We mapped workload identity to UAMI with RBAC role (SMB Elevated Contributor) on desired storage account scope, however it raises permission denied on mount attempt when storage account keys are disabled on this storage account. When we enable storage account keys it starts to work again.

What you expected to happen: CSI Driver should use RBAC role instead of downloading storage keys

How to reproduce it: Setup CSI connection to storage account with usage of workload identity -> disable storage keys on desired accounts -> try to run the application

Anything else we need to know?:

Environment:

andyzhangx commented 4 months ago

@Vegoo89 this is by design since this driver would fetch account key by workload identity, and then mount azure file by account key.

Vegoo89 commented 4 months ago

But can't it use RBAC role to authenticate? Is there any reason why it can't be done? My company is enrolling policy to disable shared keys so I am reviewing my options.

andyzhangx commented 4 months ago

@Vegoo89 that requires AAD auth first: https://learn.microsoft.com/en-us/azure/storage/files/storage-files-active-directory-overview#configure-share-level-permissions-for-azure-files, AKS nodes does not support AAD auth now.

Vegoo89 commented 4 months ago

Sorry if I am missing something but I am scratching my head now. We use workload identity on AKS for keyless auth to wide range of Azure resources.

Can't we use it in similar way to authorize ourselves to file share if UAMI has required role assigned and is present under User Assigned identities on AKS VMSS?

andyzhangx commented 4 months ago

@Vegoo89 the azure file csi driver does not support keyless auth now unless you use NFS file share, that does not require key auth.

Vegoo89 commented 4 months ago

I understand it is not supported now, however I want to understand what is the limitation and what would be necessary to work around it.

You told AKS nodes don't support AAD auth, but these are standard VMSS, managed by MS, right? If I assign UAMI to it, why can't I use it to authenticate to file share?

andyzhangx commented 4 months ago

@Vegoo89 this is all the Supported authentication scenarios for SMB file share mount: https://learn.microsoft.com/en-us/azure/storage/files/storage-files-active-directory-overview#supported-authentication-scenarios

in top level, you could assign UAMI with RBAC role (SMB Elevated Contributor) on desired storage account, but in the backend (SMB file share mount implementation details), it requires the one of the above auth, unfortunately those auth methods all requires AD domain join for aks node which is not supported now.

other context: Azure Files enforces authorization on user access to both the share level and the directory/file levels. Share-level permission assignment can be performed on Microsoft Entra users or groups managed through Azure RBAC. With Azure RBAC, the credentials you use for file access should be available or synced to Microsoft Entra ID. You can assign Azure built-in roles like Storage File Data SMB Share Reader to users or groups in Microsoft Entra ID to grant access to an Azure file share.

https://learn.microsoft.com/en-us/azure/storage/files/storage-files-active-directory-overview#access-control

andyzhangx commented 4 months ago

in brief, there is no workaround for keyless auth since this driver only supports key auth(called NTLM auth instead of kerberos auth) for smb mount unless you use NFS file share: https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/3162e05a0036759c1313d31b83c830d3ab625be6/pkg/azurefile/nodeserver.go#L343

Vegoo89 commented 4 months ago

Thanks a lot for detailed explanation. May I keep it opened until smb mount supports AAD or it won't happen in nearest future?

djsly commented 4 months ago

@andyzhangx am I understanding that with NFS, we can disable Key Access ? We are also being asked to disable Allow storage account key access on ALL Storage Account.

1- I don't see a option for disabling that even if using NFS correct ?

andyzhangx commented 4 months ago

@andyzhangx am I understanding that with NFS, we can disable Key Access ? We are also being asked to disable Allow storage account key access on ALL Storage Account.

1- I don't see a option for disabling that even if using NFS correct ?

@djsly yes, you could disable account key access if you are only using nfs file share. We will add such options for account created by this driver.

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

MikeKlebolt commented 1 week ago

/remove-lifecycle stale