Overriding a module without specifying a version can cause an error on initialization

ArnyDnD commented 1 year ago

Terraform Version

1.3.5

Terraform Configuration Files

Assuming a fiction module hosted on any Terraform Module hosting with at least two versions, let's assume 1.0.0 and 1.4.0 just for this example

main.tf

module "test" {
  source = "domain.example.com/group/module-example/example"
  version = "1.0.0"

  input = "hi"

main_override.tf

module "test" {
  input = "hello"
}

Debug Output

2022-11-29T08:34:38.479Z [INFO] Terraform version: 1.3.5 2022-11-29T08:34:38.479Z [DEBUG] using github.com/hashicorp/go-tfe v1.9.0 2022-11-29T08:34:38.479Z [DEBUG] using github.com/hashicorp/hcl/v2 v2.15.0 2022-11-29T08:34:38.479Z [DEBUG] using github.com/hashicorp/terraform-config-inspect v0.0.0-20210209133302-4fd17a0faac2 2022-11-29T08:34:38.479Z [DEBUG] using github.com/hashicorp/terraform-svchost v0.0.0-20200729002733-f050f53b9734 2022-11-29T08:34:38.479Z [DEBUG] using github.com/zclconf/go-cty v1.12.1 2022-11-29T08:34:38.479Z [INFO] Go runtime version: go1.19.3 2022-11-29T08:34:38.479Z [INFO] CLI args: []string{"/usr/bin/terraform", "init"} 2022-11-29T08:34:38.479Z [TRACE] Stdout is a terminal of width 202 2022-11-29T08:34:38.479Z [TRACE] Stderr is a terminal of width 202 2022-11-29T08:34:38.479Z [TRACE] Stdin is a terminal 2022-11-29T08:34:38.479Z [DEBUG] Attempting to open CLI config file: /root/.terraformrc 2022-11-29T08:34:38.479Z [INFO] Loading CLI configuration from /root/.terraformrc 2022-11-29T08:34:38.480Z [DEBUG] ignoring non-existing provider search directory terraform.d/plugins 2022-11-29T08:34:38.480Z [DEBUG] ignoring non-existing provider search directory /root/.terraform.d/plugins 2022-11-29T08:34:38.480Z [DEBUG] ignoring non-existing provider search directory /root/.local/share/terraform/plugins 2022-11-29T08:34:38.480Z [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins 2022-11-29T08:34:38.480Z [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins 2022-11-29T08:34:38.480Z [INFO] CLI command args: []string{"init"} Initializing modules... 2022-11-29T08:34:38.481Z [TRACE] ModuleInstaller: installing child modules for . into .terraform/modules 2022-11-29T08:34:38.481Z [DEBUG] Module installer: begin test 2022-11-29T08:34:38.485Z [TRACE] ModuleInstaller: Module installer: test 1.4.0 already installed in .terraform/modules/test 2022-11-29T08:34:38.485Z [TRACE] modsdir: writing modules manifest to .terraform/modules/modules.json There are some problems with the configuration, described below.

The Terraform configuration must be valid before initialization so that Terraform can determine which modules and providers need to be installed. ╷ │ Error: Module version requirements have changed │ │ on main.tf line 2, in module "test": │ 2: source = "domain.example.com/group/module-example/example" │ │ The version requirements have changed since this module was installed and the installed version (1.4.0) is no longer acceptable. Run "terraform init" to install all modules required by this │ configuration. ╵

Expected Behavior

After executing terraform init, I would expect to download the module "test" with version "1.0.0".

Actual Behavior

Version "1.4.0" is downloaded during terraform init and after that an error appears showing "Error: Module version requirements have changed" specifying that line 2 on main.tf has modified the module version and that terraform init should be executed. If executing terraform init again, the error persists.

Steps to Reproduce

terraform init

Additional Context

What I understand is that the module overriding on main_override.tf, as is not specifying a version, is actually setting the version "latest", creating an override at is.

While this could make some logic, I would arguably say that is quite missleading, maybe this could be fixed by just documenting this behavior?

Finally, please kindly note that although this example uses a fictionary module, I've managed to reproduce this behaviour with modules hosted on Terraform Registry, just to give an example the AWS VPC module is a good candidate to try quickly to reproduce this.

References

No response

kmoe commented 1 year ago

Thanks for the issue. I'll need to look into this further to see what behaviour is actually intended here, but the error message is at the very least quite misleading, and the behaviour doesn't seem to match the rules for merging documented at https://developer.hashicorp.com/terraform/language/files/override.

MaxymVlasov commented 1 year ago

Exists in 1.4.5, just checked

brandon-fryslie commented 1 year ago

The workaround is to specify the version in the override file. If you don't specify the version in the override file, it seems to always install the latest version available.

Our use case is restricted offline environments where we need to specify a different URL that isn't available outside of those environments. Since there isn't any way to use a variable in a module source, we need to user override files.

We generate the override files as an initial step in an Atlantis workflow, and I can confirm that specifying the version leads to the correct behavior

hashicorp / terraform