a way to initialize modules and providers while ignoring stored backend configuration

[aaronmell]

Terraform Version

Terraform v1.5.4
on darwin_arm64

Terraform Configuration Files

  required_version = ">= 1.5.4"
  backend "s3" {
    encrypt = true
  }

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.11.0"
    }
  }
}

Debug Output

Snippit of relavant debug logs

2023-08-23T10:03:43.584-0400 [INFO]  CLI command args: []string{"init", "-reconfigure", "-backend=false"}
2023-08-23T10:03:43.586-0400 [TRACE] Preserving existing state lineage "f25705b8-23d5-ed76-22a5-9ae636479294"
2023-08-23T10:03:43.586-0400 [TRACE] Preserving existing state lineage "f25705b8-23d5-ed76-22a5-9ae636479294"
2023-08-23T10:03:43.586-0400 [TRACE] Meta.Backend: working directory was previously initialized for "s3" backend
2023-08-23T10:03:43.587-0400 [INFO]  AWS Auth provider used: "SharedCredentialsProvider"
2023-08-23T10:03:43.587-0400 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2023-08-23T10:03:43.587-0400 [DEBUG] [aws-sdk-go] DEBUG: Request sts/GetCallerIdentity Details:

Expected Behavior

The documentation states that -reconfigure Reconfigure a backend, ignoring any saved configuration. -backend=false Disable backend or Terraform Cloud initialization for this configuration and use what was previously initialized instead.

I would expect that when running the init command with both of those flags, it would ignore the existing saved configuration and re-initialize the root module with no backend

Actual Behavior

The command attempts to connect to the backend. If the credentials are expired it fails.

Steps to Reproduce

1. terraform init -reconfigure -backend=false

Additional Context

No response

References

No response

[jbardin]

Hi @aaronmell,

Thanks for filing the issue. Can you explain what the end goal is here? The -reconfigure option is when you want to reconfigure the stored backend configuration without using that stored configuration at all. The -backend=false option is used when you want to fetch providers and modules without [re]initializing the backend in the first place. Terraform cannot ignore the backend configuration entirely, what is the result you expect here?

Thanks!

[jbardin]

OK, I think I was confused by the addition of -reconfigure here, the goal seems to be to ignore the stored configuration. The key part of the -backend=false help text is "use what was previously initialized instead". This was not meant to be combined with -reconfigure, but I could see how the combined help text of those options could be read that way.

The exiting behavior is working as designed, but we can look into allowing this combination as well, though perhaps another way since the word "reconfigure" does imply that reconfiguration would happen rather than ignoring it altogether.

[aaronmell]

So the intent here was I am writing a pre-commit hook that runs terraform validate. I managed to solve it another way by just checking if a .terraform folder already exists in the root module, and if it does skip running init.

[tekumara]

I too was expecting terraform init -reconfigure -backend=false would do the equivalent of:

rm .terraform/terraform.tfstate && terraform init -backend=false

ie: reinitialise terraform without a backend. This is useful when there is an existing backend and it requires authentication, but we don't want to plan or apply we just want to validate (and so don't need auth).

[tekumara]

Related to https://github.com/hashicorp/terraform/issues/29625

[apparentlymart]

Hi all,

A typical way to run validate in a scenario where backend credentials are unavailable is:

terraform init -backend=false
terraform validate

The -backend=false option instructs Terraform to skip all of the backend-related parts of terraform init. That's okay for terraform validate because it doesn't use the backend anyway; it's a config-only operation.

If you exclude -reconfigure as I've shown above, does that get the behavior you need or is something missing?

[tekumara]

If the backend has previously been initialised, then

terraform init -backend=false

will attempt to auth using the existing backend, rather than ignore it.

This is most obvious when using an AWS backend when AWS SSO creds have expired, eg:

1. authenticate to AWS SSO
2. terraform init - initialise with AWS backend
3. rm ~/.aws/sso/cache - remove SSO session token

4.   ❯ terraform init -backend=false
     Initializing modules...
     ╷
     │ Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.
     │ 
     │ Please see https://www.terraform.io/docs/language/settings/backends/s3.html
     │ for more information about providing credentials.
     │ 
     │ Error: SSOProviderInvalidToken: the SSO session has expired or is invalid

[apparentlymart]

Thanks for that existing context @tekumara. I hadn't understood that you are trying to initialize a directory where the backend was previously initialized already.

Indeed, that isn't something Terraform supports today, although I must admit I'm not sure if that omission was an intentional design decision or just an accidental consequence of the implementation; the -backend=false option was intended for situations like running terraform validate in a CI system that never initializes the backend, rather than for "de-initializing" an already-initialized backend in an existing working directory.

One avenue we could potentially explore here is to change -backend=false so that it skips all actions related to the backend, even if there's already an initialized backend in the working directory. I don't know yet what the consequences would be of doing that; we'd need to go explore the code and understand why the backend verification is happening in this codepath in the first place.

I suspect what's going on is that terraform init is trying to retrieve the state to see if there are any additional required providers there that aren't captured in the current configuration. If so, deciding that -backend=false means to ignore the backend entirely then implies that -backend=false also means to ignore any state-only provider dependencies, which would be fine if the goal is only to run terraform validate since that doesn't rely on information from the state anyway.

The combination of both -reconfigure and -backend=false at the same time doesn't really make sense, because we can't both reconfigure the backend and skip configuring the backend at the same time. I think we should consider making that combination an error that explains why that combination of options isn't supported, although we'd first need to convince ourselves that it's unlikely that anyone would be depending on the current emergent behavior of that combination for some real purpose or else we'd be breaking the Terraform v1.x compatibility promises.

(If you do want to achieve this "forget what you know about the backend" behavior with today's Terraform, one way to do that would be to delete the .terraform/terraform.tfstate file where Terraform tracks which backend is initialized. However, the existence and purpose of that file are an implementation detail rather than a promised interface, so just deleting that file won't necessarily be sufficient in future Terraform versions if the implementation of the backend model were to change.)