Cannot override a module block's "source" attribute to a local path if it has a version attribute set

[c0sco]

Hi, I have a use case where I'd like to override a module block to change it's source attribute from a registry URL to a local path. This is to allow for a custom testing setup (swapping out the registry's copy of the module with a local one in a build pipeline). I'm trying to accomplish this by creating a main_override.tf file that overrides the module definition and points the source to the local copy being tested. When doing this and running terraform init, I get the following error:

There are some problems with the configuration, described below.

The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.

Error: Module version requirements have changed

  on main_override.tf line 2, in module "vpc":
   2:   source = "./.."

The version requirements have changed since this module was installed and the
installed version is no longer acceptable. Run "terraform init" to install all
modules required by this configuration.

If I delete the version attribute out of the module block, the override is successful.

Terraform Version

Tested 0.13, 0.14 and latest clone from the master branch.

$ terraform version
Terraform v0.15.0-dev

Terraform Configuration Files

To reproduce: main.tf (the module being used can be anything, only using vpc as an example)

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.64.0"
}

main_override.tf

module "vpc" {
  source = "./.."
}

Expected Behavior

I would expect Terraform to realize it's overriding a module source and either ignore the version attribute, or allow for a mechanism to remove the attribute via the override. Alternatively I would expect an error saying that this use case isn't supported (although it seems to be as long as a version attribute isn't present).

Actual Behavior

An error occurs on init saying "Module version requirements have changed"

Steps to Reproduce

Using the code above, run terraform init. If you delete the version attribute out of main.tf and run init, no error occurs.

[apparentlymart]

Hi @c0sco! Thanks for sharing this use-case.

The Override Files feature is, unfortunately, a pretty old addition to Terraform that wasn't designed particularly thoroughly when it was introduced, and so a lot of its design just "is what it is" and we've maintained the behaviors as best we can over time. For that reason, we tend to consider whatever it does today to be "working as designed" and would consider changes to that behavior to be a feature request, and so I'm going to relabel this issue with that in mind just because that means we'll use it with our process for designing new language changes rather than our process for fixing situations where the implementation disagrees with the design.

With that said, the use-case you've shared (of overriding a remote module to a local directory temporarily for testing) does make sense to me. If we didn't already have this old override files mechanism and you'd shared the use-case of overriding a module for testing then I expect we would not try to solve the use case with override files, but since the override files mechanism already exists it does give us one possible angle to address this need.

If we do try to solve this by changing the override files design, I could see the argument that source and version function as if they were a single argument really, and so it would make sense for them to override together: if an override file sets source without setting version then that means to override version to not be set at all. The opposite is a bit more debatable though: overriding version without overriding source can make sense as long as source was already set to a registry-style source address, but would be invalid if it were any other kind of source.

---

In the project that led to Development Overrides for Provider Developers we'd initially designed a different mechanism involving creating a dependency-specific override file in the local directory -- something like .terraform.override.hcl to go along with .terraform.lock.hcl -- and adding some commands to help manage that file for temporary development overrides of both providers and modules.

We ended up having to cut that part of the project due to deadlines, which is why the documentation for dev_overrides currently ends with a disclaimer about the mechanism potentially changing in future: the current dev_overrides design was a compromise that was easier to fit in with what was already implemented, but is non-ideal in that it serves as a global override rather than a local override.

With that said then, another way to meet this use-case would be for us to move forward with the idea of a dependency-specific override mechanism. I expect the result would be more ergonomic because it would be designed with module overriding in mind -- for example, it ought to allow you to override modules called by your child modules too, without having to modify them directly.

It would also have the advantage of not changing the existing override behavior. Although on the surface it seems like a small change to the override behavior wouldn't hurt here, previous experience showed us that there are various folks depending on specific quirks of the current overrides design such that changes to it often have unintended consequences. A new mechanism focused only on this problem would avoid that risk.

[mcfedr]

Just found this today, having a similar issue, where I found the docs for dev_overrides but unfortunately it doesn't work for modules. Figure I'll add a comment as a little bump on a 2 year old issue ))

Guess I'll stick with replacing all the sources for now

[ialexj]

Just to add another use-case, I came across this issue while using "terraform test" in a pipeline in conjunction with Terraform Private Registry examples.

Our private Registry modules usually include a couple of examples for common use-cases (the ones that show up in the Examples dropdown in the Private Registry UI), and we've also decided to add some tests to the examples to document our expectations.

We also want to be able to run "terraform test" in our module PR pipeline, which includes the example tests. This is all to ensure that any changes to the module include updated examples and tests for all PRs.

Where this goes wrong is that, in the examples, we need to use the private registry source for the module, as otherwise the example can't be applied with Terraform Cloud, and because we want our devs to be able to just copy-paste from the example. This causes a chicken-and-egg problem, where now the example tests always run against the version that's already on the registry, and not the version that's currently being PRed.

So the need here is to be able to override the module source from the registry to a local relative path, namely ../.. (since our examples are in /examples/my-example-1/...), so that the examples always test against the new version of the code.

I was hoping that maybe Terraform has some built-in means to accomplish this, but failing that, we'll be manually rewriting the source files before running the tests, and reverting after.