Unable to override `profile` using `AWS_PROFILE` environment variable

glindstedt commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.4
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v3.73.0

Affected Resource(s)

Provider and backend configuration.

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

{
  terraform: {
    backend: {
      s3: {
        bucket: 'some-state-bucket',
        region: 'eu-north-1',
        key: 'aws/',
        profile: 'some-profile',
      },
    },
  },
  provider: [
    {
      aws: {
        region: 'eu-north-1',
        profile: 'some-profile',
        allowed_account_ids: ['xxxxxxxxxxxx'],
      },
    },
  ],
}

Debug Output

2022-01-25T13:42:49.715+0100 [INFO]  Terraform version: 1.1.4
2022-01-25T13:42:49.715+0100 [INFO]  Go runtime version: go1.17.2
2022-01-25T13:42:49.715+0100 [INFO]  CLI args: []string{"/home/glindstedt/.cache/bazel/_bazel_glindstedt/88dc742fbc85c9bfde9575f9abe9fe92/execroot/embark/bazel-out/k8-fastbuild/bin/infra/aws/terraform.runfiles/embark/bazel/terraform-bin", "init"}
2022-01-25T13:42:49.715+0100 [DEBUG] Attempting to open CLI config file: /home/glindstedt/.terraformrc
2022-01-25T13:42:49.715+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory /home/glindstedt/.terraform.d/plugins
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory /home/glindstedt/.local/share/terraform/plugins
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/pop/terraform/plugins
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory /home/glindstedt/.local/share/flatpak/exports/share/terraform/plugins
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory /var/lib/flatpak/exports/share/terraform/plugins
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2022-01-25T13:42:49.715+0100 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2022-01-25T13:42:49.715+0100 [INFO]  CLI command args: []string{"init"}
2022-01-25T13:42:49.716+0100 [DEBUG] New state was assigned lineage "53d73a4f-089e-f9b7-32c3-374ac128105e"
2022-01-25T13:42:49.716+0100 [DEBUG] checking for provisioner in "."
2022-01-25T13:42:49.716+0100 [DEBUG] checking for provisioner in "/home/glindstedt/.cache/bazel/_bazel_glindstedt/88dc742fbc85c9bfde9575f9abe9fe92/execroot/embark/bazel-out/k8-fastbuild/bin/bazel"
2022-01-25T13:42:49.717+0100 [INFO]  Attempting to use session-derived credentials
╷
│ Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.
│
│ Please see https://www.terraform.io/docs/language/settings/backends/s3.html
│ for more information about providing credentials.
│
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│       For verbose messaging see aws.Config.CredentialsChainVerboseErrors
│
│
│
╵

Expected Behavior

That AWS_PROFILE is able to override the profile set in the terraform file. In general I expect programs to treat environment variables (and command line arguments) as overrides to options which might otherwise be defined in configuration files.

Actual Behavior

Terraform still tries to use the profile that's set in the terraform file. If I comment out that line it correctly picks up on the AWS_PROFILE environment variable and uses that profile.

Steps to Reproduce

Define backend and provider with profile set to a profile that doesn't exist in your config
Set the AWS_PROFILE environment variable to a profile that does exist
Run terraform plan, terraform will insist on trying to use the profile defined in the terraform file instead of accepting the override

gdavison commented 2 years ago

Hi @glindstedt, thanks for submitting this.

Values directly configured in the Terraform source files have the highest precedence when resolving configuration. The order is:

Hardcoded values in the provider configuration block, i.e. provider "aws" { ... }
Environment variables
The AWS shared credentials and configuration files

While we refer to Terraform files as a "configuration", it's best to think of them as source files, and not configuration files. Directly configuring the profile value in the provider block is equivalent to passing --profile when using the AWS CLI.

There are several solutions you could use:

Always specify the profile using AWS_PROFILE
Make the profile parameter a Terraform variable, so that it can be configured from the command line or a variable file

github-actions[bot] commented 2 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws