Support Workload Identity Federation (OIDC) in CI/CD Pipelines

[bravous]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

Currently when running pulumi (gcp) in an automated pipeline we need to provide a downloaded JSON key via the GOOGLE_CREDENTIALS environment variable. This is not a best practice. It is better to use Workload Identity Federation, this will make sure no (long lived) keys are downloaded but short lived tokens are generated for authentication.

Service account keys could pose a security risk if compromised. Therefor it is recommended to avoid downloading service account keys and instead use the Workload Identity Federation. More can be found here on the best way to authenticate service accounts on Google Cloud here.

An example bitbucket-pipeline.yaml using OIDC and Workload Identity Federation would look like this:

image: pulumi/pulumi-nodejs:latest

pipelines:
  master:
      - step:
          name: Infrastructure Update 
          oidc: true
          caches:
            - node
          script:
            - echo $BITBUCKET_STEP_OIDC_TOKEN > /tmp/oidc-token.txt
            - export GOOGLE_CREDENTIALS=${PWD}/credential-config.json
            - yarn install 
            - pulumi -s ${PULUMI_STACK_NAME} --refresh

The credential-config.json file provides the hints on how to get the token to the google client libraries . And the oidc = true makes sure bitbucket generates an OIDC token to use in the OIDC flow.

Currently when we run this pipeline we get permission denied errors, it looks like it tries to use the credential-config.json file as a real/regular JSON key to authenticate and does not follow the OIDC flow using the information.

The errors we see in the log file look as follow:

Diagnostics:
  gcp:serviceAccount:IAMBinding (serviceAccount-bitbucket-binding):
    error: Preview failed: refreshing urn:pulumi:master::workload-federation::gcp:serviceAccount/iAMBinding:IAMBinding::serviceAccount-bitbucket-binding: 1 error occurred:
        * Error when reading or editing Resource "'**************'" with IAM Binding (Role "roles/iam.workloadIdentityUser"): Error retrieving IAM policy for service account '**************': Post "'**************?alt=json&options.requestedPolicyVersion=3&prettyPrint=false": oauth2/google: status code 403: {
      "error": {
        "code": 403,
        "message": "The caller does not have permission",
        "status": "PERMISSION_DENIED"
      }
    }

When we replace the config file with a real JSON key it works. So it looks like the underlying library used for authentication is not able to utilize workload identity flow.

Affected area/feature

Don't really know but it looks like it should be part of the underlying googel client library that is responsible for authentocating.

[roothorp]

Hi @bravous, thanks for the issue! I've raised this with the team so it can be prioritized and added to our work stack.

[bravous]

Hi @roothorp , after creating everything again from scratch (could not believe it would not work), it worked so I think i messed something up somewhere. Thanks for taking a look, you can close this issue. Maybe change it in updating documentation to basically promote identity federation as best practice ;-)

[mvarchdev]

Hello. Is this implemented in pulumi Cloud pipelines?