ArgoCD connect to Google Cloud Source Repo from GKE Using WorkloadIdentity credentials

[Vavinash-github]

Currently ArgoCD supports connecting to Google Cloud Source Repo only using Google Service Account (GSA) Private key Json file mentioned in --> https://argo-cd.readthedocs.io/en/stable/user-guide/private-repositories/#google-cloud-source

Storing the GSA in kubernetes cluster is not secure and is not approved in many organizations.

Since WorkloadIdentity can be used in GKE clusters Argocd Deployments should be able to use the KSA (Kubernetes Service account) and run as GSA user using WorkloadIdentity and connect to Google Cloud Source Repo.

Using WorkloadIdentity improves the security because there are no private key files stored in GKE cluster and the credentials are short lived and will be auto rotated.

I would like to know when this feature can be made available or any alternate approaches using WorkloadIdentity to connect to Google Cloud Source Repo.

Thanks..

[torfjor]

This is already supported, using External Account Credentials (albeit not very well documented). This method will work for GKE and any other external cluster (although you have to set up the workload identity pool manually in this case).

Assuming your setup is GKE:

1. Patch your argocd-repo-server to mount the KSA token with the cluster workload identity pool as audience:

   - patch: |
     - op: add
       path: /spec/template/spec/volumes/-
       value:
         name: gcp-ksa
         projected:
           defaultMode: 420
           sources:
           - serviceAccountToken:
               path: token
               audience: YOUR_PROJECT_ID.svc.id.goog
               expirationSeconds: 172800
     - op: add
       path: /spec/template/spec/containers/0/volumeMounts/-
       value:
         name: gcp-ksa
         mountPath: /var/run/secrets/tokens/gcp-ksa
         readOnly: true
   target:
     kind: Deployment
     name: argocd-repo-server

2. Use the following JSON contents for the service account key:

   {
   "type": "external_account",
   "audience": "identitynamespace:YOUR_PROJECT_ID.svc.id.goog:https://container.googleapis.com/v1/projects/PROJECT_ID/locations/CLUSTER_REGION/clusters/CLUSTER_NAME",
   "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/TARGET_SERVICE_ACCOUNT@YOUR_PROJECT_ID.iam.gserviceaccount.com:generateAccessToken",
   "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
   "token_url": "https://sts.googleapis.com/v1/token",
   "credential_source": {
   "file": "/var/run/secrets/tokens/gcp-ksa/token"
   }
   }

3. Set up IAM bindings to the roles/iam.workloadIdentityUser role with serviceAccount:YOUR_PROJECT_ID.svc.id.goog[argocd/argocd-repo-server] as member on TARGET_SERVICE_ACCOUNT.

[Vavinash-github]

Hi Torfjor,

Thanks for the article above.. I was able to successfully see a connection on the repo page in argocd UI.

But I am facing issue when syncing the Google source repository after creating an Argocd Application.

ERROR:

Git fetch origin --tags --force --prune failed exit status 128: remote:PERMISSION_DENIED: The caller does not have permission remote: [type.googleapis.com/google.rpc.RequestInfo] remote: request_id: "hash" fatal: Authentication failed for GCS repo

[Vavinash-github]

I am successfully able to connect to repository using both UI and CLI but when I create an Application The path selector does not show any suggestions and adding the application fails.

[torfjor]

It looks like the authentication code path for GCS repos makes some assumptions about the shape of the JSON credential you are passing in:

https://github.com/argoproj/argo-cd/blob/75b438a27884bea61bda1cfe3d00d0781ba1d279/util/git/creds.go#L511

You can fix that by supplying a non-empty, client_email field in the JSON credential. Its value can be whatever.

I'll let the project maintainers chime in on whether we want to fix this, or just document that authentication with Workload Identity can be done using External Account Credentials for GKE and remote clusters (Anthos Fleet / GKEHub), as long as you use the workaround I mentioned above ☝️ .