Missing information on how to override backend endpoints from command line

[oto-macenauer-absa]

Terraform Version

Terraform v1.7.1
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.20.1
+ provider registry.terraform.io/hashicorp/kubernetes v2.23.0
+ provider registry.terraform.io/hashicorp/random v3.6.0

Your version of Terraform is out of date! The latest version
is 1.7.2. You can update by downloading from https://www.terraform.io/downloads.html

Affected Pages

https://developer.hashicorp.com/terraform/language/settings/backends/s3#endpoint

What is the docs issue?

Missing information on how to override endpoints.s3 and similar using command line arguments

I tried this:

terraform init \   
  -backend-config="endpoints.s3=http://127.0.0.1:9000" \                 
  -input=false -reconfigure

Initializing the backend...
Initializing modules...
╷
│ Error: Invalid backend configuration argument
│ 
│ The backend configuration argument "endpoints.s3" given on the command line is not expected for the selected
│ backend type.
╵

but wasn't successful. The deprecated parameter "endpoint" can't be used as it's not possible to use it together with already existing endpoints.s3

Proposal

No response

References

No response

[oto-macenauer-absa]

The correct way to specify the override is using this parameter:

-backend-config='endpoints={s3="http://127.0.0.1:9000"}'

[apparentlymart]

Thanks for reporting this, @oto-macenauer-absa!

In case it's useful to someone who works on this issue in future, I'll note that the documentation about "overriding" parts of the backend configuration is in Partial Configuration, because that was the terminology we originally used when first introducing the idea that all arguments in a backend block are optional as long as they are specified in one of these other ways. I'm not sure that terminology really resonates with how people think of that feature. For example, this issue reasonably uses the word "override" to describe it.

When we originally wrote the documentation there weren't yet examples of backend configuration arguments that were of collection or structural types, so the section on command-line overrides only describes how to set primitive-typed arguments, and doesn't say anything about how to assign complex-typed values.

In practice the syntax here is the same as setting input variables on the command line, which does include some examples of how to assign to a list-typed or map-typed input variable. Maybe we can find some way to unify these somewhere so that it's clearer that the patterns are the same.

Finally, I notice that the S3 backend's Configuration section mentions -backend-config in passing but doesn't prominently link to Using a Backend Block to connect with the general information about how backend configuration works. I guess this was on the assumption that readers would find the general documentation first and only then read the backend-type-specific pages, but of course folks are just as likely to end up on the S3 backend page first, if e.g. they already knew that was what they needed to use from reading something elsewhere on the internet.

[oto-macenauer-absa]

Hi, once again, thanks for the answer @apparentlymart!

I must admit I have skipped the setting input variables on the command line. We're using var files to provide variables to terraform so that's why I didn't get familiar with the command line arguments. That's why I thought it would be useful to put information also to the backend configuration.

On the other hand this pointed me out to tfbackend files for configuration, so it might be better idea to switch to those.

As for the terminology, the term "overrides" is used by helm documentation so that's why it may have stuck in my mind.

Thank you for the explanation.