search

ofek / csi-gcs

Kubernetes CSI driver for Google Cloud Storage
https://ofek.dev/csi-gcs/
Apache License 2.0
153 stars 39 forks source link

Allow use of k8s service accounts and workload identity for auth #127

Open elibixby opened 2 years ago

elibixby commented 2 years ago

The driver should likely be configured with a k8s service account (either in PVC config or storage class config) that has permissions to the bucket via Workload Identity rather than using a key.

Details here: https://kubernetes.io/blog/2020/12/18/kubernetes-1.20-pod-impersonation-short-lived-volumes-in-csi/#using-gcp-apis

sshcherbakov commented 2 years ago

+1 The Cloud SDK libraries support using Workload Identity for authentication by either contacting local metadata server when running on GCE or GKE or by accepting fleet workload identity configuration in the GOOGLE_APPLICATION_CREDENTIALS environment variable instead of the json file with service account key as described here (please see step 6 for example).

Similar Google Secret Manager CSI driver supports workload identity when running on GKE (and in Anthos clusters outside of GCP hopefully soon), not relying on the Cloud SDK client libraries though.

ofek commented 2 years ago

We'd happily accept a PR 🙂

lancehudson commented 2 years ago

This doesnt work with hostNetwork. I can however confirm that if you disable hostNetworking in the DaemonSet that workload identity starts working.

lemaral commented 2 years ago

Is hostNetwork necessary then? It seems to work without

sshcherbakov commented 2 years ago

I am not sure how hostNetwork is related to WIF. WIF introduces additional credential type "external_account" for the client authentication libraries to be supported. The implementation would need to make a couple of calls to the endpoints defined in the config file. I don't think that setting hostNetwork to any value would remove the need in the WIF authentication flow calls.

sshcherbakov commented 2 years ago

When it "works", are you sure that you are authenticating as "workload" (serviceAccount:FLEET_PROJECT_ID.svc.id.goog[K8S_NAMESPACE/KSA_NAME]) rather than a single GSA, that the underlying GCE VM is running with?

lancehudson commented 2 years ago

Ill dig through the audit logs but the nodes dont have access (it was failing before i disabled hostnetworking)

Doesnt the library you are using for gcs automatically use the metadata service if no credentials are specified?

sshcherbakov commented 2 years ago

It does, but 1) it depends on the environment where you are, e.g. GCE, GKE, non-GCP 2) the point of WIF is to work seamlessly (similarly) in environments outside of GCP (on-prem, VMware, AWS, Azure, etc) where metadata server (neither GCE nor GKE one) is not available and simultaneously not to require GSA keys for authentication.