Terraofrm asks to migrate when backend config changes remote location

[aaronbrashears]

Terraform Version

Terraform v0.11.11
+ provider.aws v1.60.0
+ provider.template v2.0.0

Terraform Configuration Files

terraform {
  backend "s3" {
    // filled in by -backend-config to `terraform init`
    // https://www.terraform.io/docs/backends/config.html#partial-configuration
  }
}

Expected Behavior

Terraform should use new the backend config provided to terraform init which has changed since the last terraform apply

Actual Behavior

Terraform emits the message:

Initializing the backend...
Backend configuration changed!

Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.

Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "s3" backend to the
  newly configured "s3" backend. An existing non-empty state already exists in
  the new backend. The two states have been saved to temporary files that will be
  removed after responding to this query.

  Previous (type "s3"): /var/folders/5p/wg925zw540v3qj6ttm9kk2xmpvjx48/T/terraform221171925/1-s3.tfstate
  New      (type "s3"): /var/folders/5p/wg925zw540v3qj6ttm9kk2xmpvjx48/T/terraform221171925/2-s3.tfstate

  Do you want to overwrite the state in the new backend with the previous state?
  Enter "yes" to copy and "no" to start with the existing state in the newly
  configured "s3" backend.

Steps to Reproduce

1. terraform init -backend-config="bucket=bucket1" -backend-config="key=key1" -backend-config="profile=profile1" -backend-config="region=us-west-2"
2. terraform apply
3. terraform init -backend-config="bucket=bucket2" -backend-config="key=key2" -backend-config="profile=profile2" -backend-config="region=us-west-2"
4. terraform apply

Additional Context

This is useful to have exactly the same terraform used on different accounts using remote state.

[apparentlymart]

Hi @aaronbrashears,

I think the -reconfigure option to terraform init will get the behavior you want here. It acts approximately as if you'd answered "no" to that interactive prompt, discarding the previous backend configuration and activating the new one fresh without any migration.

[aaronbrashears]

I have tested -reconfigure and it works splendidly in our use case.

[iayongwa]

Hello @apparentlymart, When using Terraform in an Azure DevOps YAML pipeline, trying to pass 'init -reconfigure' flag under command fails for me when I try to run the pipeline with message: peError: provider[command] is not a function Running a pipeline in Azure DevOps means I don't have that interactive input option to input a 'yes' or 'no' value so that my state file can be start on a new slate. Any ideas on how to go around this?

[ChamMach]

Same issue @isambay but with GitHub actions

Does -reconfigure really works ?

[rubsTerri]

I have the same issue here, and i hace not found any option to correct this behavior in azure devops

[cezar-gradinariu]

Same here

[sbhattach]

Same issue @isambay but with GitHub actions

Does -reconfigure really works ?

Try using - task: TerraformTaskV2@2

[apparentlymart]

Hi all,

The -reconfigure option I suggested before was aimed at the problem of switching from one backend configuration to another.

The most recent comments seem to be in a slightly different situation: activating a backend for the first time when you already have a local state file in your working directory. It's true that -reconfigure doesn't handle that case, because there isn't an existing backend configuration to reconfigure and so Terraform is ending up in codepath intended for switching from the legacy no-backend mode into having an explicit backend.

With that said, I don't really understand how an automated workflow would end up with a local state file in the working directory prior to running terraform init; I would expect such automation to be starting with an uninitialized working directory that contains only the configuration, and so there would be nothing to migrate. Are there some other steps running prior to terraform init that might be creating a state file? Or alternatively, have you accidentally placed a state file in your version control that would therefore be placed there by your automation's git clone step?

The expected situation for automation would be for your configuration under version control to contain a backend block and for there to be no state in your local working directory, because you're using remote state and therefore the state lives in whatever remote system you configured. I think the path forward would therefore to be to figure out why you have a local state file in your directory and fix that so that you're starting from a totally clean working directory, and so that terraform init can therefore just activate the configured backend without trying to migrate you from the legacy no-backend situation.