Remote backend token is ignored by global setting.

[alexmherrmann]

Terraform version

terraform version 0.12

The problem

When using a .terraformrc with credential block and a terraform module with remote backend and different token, e.g.

terraform {
    backend "remote" {
        token = "this token is ignored"
        organization = "orgname"
        workspaces {
            name = "workspacename"
        }
    }
}

The .terraformrc configuration seems to override what is set in the local .tf file. It seems like it should be the other way around.

To test

2 Terraform accounts are needed, use the token of one in the .terraformrc, and use the token of the other in a new projects remote backend configuration. Notice that as long as there is something in the .terraformrc, the token in the remote backend will not work.

Use case

My company has several terraform projects for which I am a member with a work account. I also maintain another terraform organization separate from these work accounts.

Workaround

Add something like alias terraform-0.12-mine="TF_CLI_CONFIG_FILE=~/.terraformrc.mine terraform-0.12" to your .bashrc, where .terraformrc.mine is a file with the token for your personal account.

Use terraform-0.12-mine instead of terraform when working in your own projects.

[alexmherrmann]

Actually, it appears as if the terraform remote backend token field is not working at all, the below .tf file:

terraform {
    backend "remote" {
        token        = "***"
        organization = "anorganization"
        workspaces {
            name = "aworkspace"
        }
    }
}

gives me

Initializing modules...

Initializing the backend...

Error: Required token could not be found

Make sure you configured a credentials block for app.terraform.io in your CLI
Config File.

With no .terraformrc

Am I doing something wrong or is this a separate issue?

[alisdair]

Thanks for reporting this problem! I think the root problem here is a fault in the backend cache invalidation.

When running init, we compare the backend configuration cached in state (in .terraform/terraform.tfstate) with the new backend configuration. If the two are identical, we don't reinitialize. This check may not be working correctly.

You can work around this by clearing the backend state, moving or deleting the .terraform/terraform.tfstate file. Once done, rerunning init should succeed and cache the new token value.

I'm continuing to investigate, as I haven't isolated the exact problem here, but wanted to leave this workaround in case it's useful to you (or future searchers).

[alexmherrmann]

Until I figured out I could remove .terraform to fix the problem I thought I was going crazy watching old/unrelated tokens going across the wire. Thanks for the info on only needing to remove the terraform.tfstate. Juggling 3 accounts across many backends and I'm excited for a real fix to get put into place!

[samm-git]

The root cause of it is very simple. See https://github.com/hashicorp/terraform/blob/ce638c9231b2763725014b1cc1cbb49dd334be45/internal/backend/remote/backend.go#L256-L276

Global token always takes precedence over local once and this is causing issue we have here.

[apparentlymart]

Hi all!

In the meantime since this issue was opened, the "remote" backend has been replaced by a first-class Terraform Cloud integration. That new configuration block and its associated functionality are intended to replace backend "remote". Since backend "remote" now continues to exist only for backward-compatibility, we are intending to preserve its current behavior as-is for as long as it remains available.

The new cloud block also has a token setting, and that setting does take priority over ambient credentials when set as requested in this issue. Therefore although the remote backend itself doesn't behave in this way, its new replacement does have the requested behavior and so I'm going to close this.

If you are current using backend "remote" and want to migrate to the cloud block, please refer to Migrating from the remote Backend for detailed instructions.

---

It might interest some of you to know that modern Terraform now also supports setting all of the cloud block settings and the per-hostname credentials via environment variables:

• Environment variables which act as cloud block settings
• Environment variable credentials

I would recommend using either credentials blocks in the CLI config or the environment variables credentials mechanism to set your credentials for Terraform Cloud, rather than setting token in the cloud block, because the credentials for app.terraform.io (or your own Terraform Enterprise hostname) are applicable to the private module registry and private provider registry features too, and so setting your credentials in the more general location will allow Terraform to use the same token in all of those cases. The token argument in the cloud block is only for state storage and remote operations.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.