Terraform init with gcs module using GOOGLE_OAUTH_ACCESS_TOKEN

[AurelienBegou]

Hi,

We are trying to secure CI/CD pipeline using HashiCorp Vault with the roleset "access_token" configuration (https://www.vaultproject.io/docs/secrets/gcp#examples) and in addition we would use modules published on GCS (https://www.terraform.io/docs/language/modules/sources.html#gcs-bucket)

After some tests, it seems that the provider (https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#access_token) is taking the access token correctly with the "apply" but not on the "init" phase to be able to download the module using the access token.

Terraform Version

Terraform v1.0.4
on linux_amd64

Terraform Configuration Files

module "custom" {
  source = "gcs::https://www.googleapis.com/storage/v1/bucketName/module.zip"
}

Debug Output

Initializing modules... Downloading gcs::https://www.googleapis.com/storage/v1/bucketName/module.zip for custom... 2021-08-10T13:50:06.749Z [INFO] CLI command args: []string{"init"} 2021-08-10T13:50:06.750Z [TRACE] ModuleInstaller: installing child modules for . into .terraform/modules 2021-08-10T13:50:06.750Z [DEBUG] Module installer: begin custom 2021-08-10T13:50:06.750Z [TRACE] ModuleInstaller: custom is not yet installed 2021-08-10T13:50:06.750Z [TRACE] ModuleInstaller: cleaning directory .terraform/modules/custom prior to install of custom 2021-08-10T13:50:06.750Z [TRACE] ModuleInstaller: custom address "gcs::https://www.googleapis.com/storage/v1/bucketName/module.zip" will be handled by go-getter 2021-08-10T13:50:06.750Z [DEBUG] will download "gcs::https://www.googleapis.com/storage/v1/bucketName/module.zip" to .terraform/modules/custom 2021-08-10T13:50:06.750Z [TRACE] fetching "gcs::https://www.googleapis.com/storage/v1/bucketName/module.zip" to ".terraform/modules/custom" 2021-08-10T13:50:06.781Z [TRACE] modsdir: writing modules manifest to .terraform/modules/modules.json ╷ │ Error: Failed to download module │ │ Could not download module "custom" (test.tf:1) source code from │ "gcs::https://www.googleapis.com/storage/v1/bucketName/module.zip": dialing: google: could not find default │ credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information. ╵

Expected Behavior

Be able to download gcs modules using GOOGLE_OAUTH_ACCESS_TOKEN

Actual Behavior

No use of the variable GOOGLE_OAUTH_ACCESS_TOKEN Works only with gcs module publicly available or using GOOGLE_APPLICATION_CREDENTIALS

Steps to Reproduce

1. docker run -it --entrypoint="sh" hashicorp/terraform:1.0.4
2. apk add --update --no-cache python3 bash curl
3. CLOUD_SDK_VERSION=351.0.0
4. wget https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz -P /&& \ tar xzf /google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz -C / && \ rm /google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz
5. PATH=/google-cloud-sdk/bin:$PATH
6. export Google private key in local
7. gcloud auth activate-service-account --key-file=sa-privatekey.json
8. export GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token) (https://cloud.google.com/sdk/gcloud/reference/auth/print-access-token)
9. Test the validity of your token
10. curl -H "Content-Type: application/x-www-form-urlencoded" -d "access_token=$GOOGLE_OAUTH_ACCESS_TOKEN" https://www.googleapis.com/oauth2/v1/tokeninfo
11. curl -X GET -H "Authorization: Bearer $GOOGLE_OAUTH_ACCESS_TOKEN" "https://www.googleapis.com/storage/v1/b/bucketName/o/module.zip"
12. terraform init

Many thanks in advance for your help.

[alisdair]

I think this is an issue with the go-getter library, which would probably be addressed by this upstream PR: https://github.com/hashicorp/go-getter/pull/302