bq component fails when using Workload Identity Federation through /auth

[hugohjerten]

TL;DR

The gcloud component bq (BigQuery) fails, due to not having "valid credentials". This is after successfully authenticating using /auth with the workload_identity_provider option, and even though the regular gcloud command works as expected.

Expected behavior

I wish to move from long-lived service account keys to using Workflow Identity Federation in my Github Action Workflows. My workflows use the the gcloud tool and install the component BigQuery component (bq). After successfully authenticating using the /auth Github Action with the workload_identity_provider option, I expect the bq component to be able to properly use the generated credentials file also.

Observed behavior

After installing the bq component and authenticating using the /auth Github Action with the workload_identity_provider option, any usage of the bq command results in the following output:

ERROR: (bq) Your current active account [***@****.iam.gserviceaccount.com] does not have any valid credentials
Please run:

  $ gcloud auth login

to obtain new credentials.

For service account, please activate it first:

  $ gcloud auth activate-service-account ACCOUNT

Action YAML

name: failing-bq

permissions:
  contents: 'read'
  id-token: 'write'

env:
  WIF_PROVIDER: ${{ secret.WIF_PROVIDER }}
  SERVICE_ACCOUNT: ${{ secret.SERVICE_ACCOUNT }}

jobs:
  example:
    name: example
    runs-on: ubuntu-20.04
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - id: auth
        name: GCP authentication
        uses: google-github-actions/auth@v0
        with:
          workload_identity_provider: ${{env.WIF_PROVIDER}}
          service_account: ${{env.SERVICE_ACCOUNT}}

      - name: Setup gcloud CLI
        uses: google-github-actions/setup-gcloud@v0
        with:
          version: 366.0.0

      - name: Install bq client
        run: gcloud components install bq --quiet

      - name: Use gcloud CLI
        run: |-
          gcloud info

      - name: Testing to list service accounts
        run: gcloud iam service-accounts list

      - name: Failing step
        run: bq ls myDataset

Additional information

I have tried explicitly authenticating in gcloud also, using the generated credential files, but this made no difference:

- id: creds
   name: gcloud login
   run: gcloud auth login --brief --cred-file="${{ steps.auth.outputs.credentials_file_path }}" --quiet

When googling I have also examined the possibility of a bug (see here & here) that has to do with the flag CLOUDSDK_PYTHON_SITEPACKAGES=1. But this didn't help either...

[sethvargo]

Neither bq nor gsutil support Workload Identity Federation yet. Unfortunately there's nothing we can do in these actions to fix this.

[bharathkkb]

@sethvargo should we add this as a warning to the WI section like auth?

[sethvargo]

https://github.com/google-github-actions/setup-gcloud/pull/463

[hugohjerten]

@sethvargo aah, thanks! I had missed this entirely. Thanks for letting me know :)

[hugohjerten]

@sethvargo I have a hard time finding additional information on when support for bq & gsutil can be expected. Do you have any information/links on this? Thanks in advance.

[sethvargo]

Hi @hugohjerten - it would be up to the teams that own those tools to add support. This is usually driven by customer demand, so if you have a GCP TAM, ask them to open a bug on your behalf.

[MartinNowak]

Related, but not specific feature request. gcloud storage rsync [208889871] - Visible to Public - Issue Tracker