terraform get: can't use variable in module source parameter?

[amaczuga]

I'm trying to avoid hard-coding module sources; the simplest approach would be:

variable "foo_module_source" {
  default = "github.com/thisisme/terraform-foo-module"
}

module "foo" {
  source = "${var.foo_module_source}"
}

The result I get while attempting to run terraform get -update is

Error loading Terraform: Error downloading modules: error downloading module 'file:///home/thisisme/terraform-env/${var.foo_module_source}': source path error: stat /home/thisisme/terraform-env/${var.foo_module_source}: no such file or directory

[pixelicous]

@lorengordon I agree.. this is nonsense.. that and the fact that everytime you pull a whole repository instead of a leaf

[luisdavim]

We use this http://bensnape.com/2016/01/14/terraform-design-patterns-the-terrafile/ I think it would be reasonable to have something like that natively.

[jjshoe]

@mitchellh - It would be great if hashicorp could re-look at this. Though it's been closed, and split into two cases, which don't address all the reasons for this, it's more commented then any current open issue.

[akvadrako]

The best workaround I have found is by using putting something like this in override.tf

module "core" {
    source = "/Users/dev/terraform-modules/core"
}
...

Not ideal, but seems to work.

[MichaelDeCorte]

@mitchellh agreement with @jjshoe the original issue of allowing interpolation for the source parameter has not been addressed. This issue should be opened, or a new one forked off. But it should not be closed.

[MichaelDeCorte]

@akvadrako I'm not following your workaround. Can you elaborate?

[akvadrako]

@MichaelDeCorte It's just that it's possible to override the module source parameters with an external file. My use case is module development, where I want to replace several references to git repos with local checkouts.

[MichaelDeCorte]

@mitchellh elaborating an example to allow the for absolute paths relative to TF-Home. Assume the below directory / file structure. This is a common pattern where repo1 is a shared repository that is downloaded locally via a script as a workaround for the source interpolation issue.

TF-Home /

• app1.tf
• appSubDir1
  • app2.tf
    • repo1 /
    • foo1.tf
    • foo2.tf

Assume that app1, app2 and foo1.tf all depend on foo2. The source parameter would be: app1: repo1/foo2.tf app2: ../repo1/foo2.tf foo1: foo2.tf

I hope it's clear that its not great.

[pixelicous]

@akvadrako Guys the best method to get around it is to wrap your terraform in a script.

Our powershell wrapper does so many things to over come terraform restrictions, we cant use terraform without, basically we did something like the guys in terragrunt did, plus many more addons on it, i cant understand how somebody can even use terraform as is out of the box without some interpolation in those missing places..

anyhow, i really hope hashicorp will decide to change some parts of the product, because it is really constricting, some of those things should have been thought of much before

[luisdavim]

Yeah, we've been using the Terrafile approach (see my comment above) it works pretty well but it forces us to use a wrapper script, I think that the Terrafile pattern should be supported by Terraform. This could easily be added to the get phase.

[nikolay]

The rationale to disallow this so that intelligent people can't download random modules is the same as not having a division operator as somebody may decide to divide by zero one day.

[tuusberg]

+1. Changing module versions manually is error prone.

[midacts]

How else can you do it. :o

On Sat, Oct 20, 2018, 10:17 AM Matthew Tuusberg notifications@github.com wrote:

+1. Changing module versions manually is error prone.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hashicorp/terraform/issues/1439#issuecomment-431585637, or mute the thread https://github.com/notifications/unsubscribe-auth/ADxtkMTqJSkZ98V__pZRc_eVZVqyMbZfks5umzBjgaJpZM4D9Dyw .

[richardgavel]

There is a similar issue in not being able to use interpolation syntax when providing configuration for back ends (say S3 bucket/region). You get around that by using terraform init -backend-config so that value is known at the beginning of the lifecycle. Couldn't something be done similarly (provide the value as some kind of command line param)?

[discreet]

This is something I've been wanting for a while and have been thinking a lot about. The use case I have is I wrote a bunch of terraform code to deploy a kubernetes cluster. In my code I have a variables module which lives in a git repo and contains all my input variables based on region and environment. This allows me to use the same exact code to deploy my kubernetes cluster to multiple AWS account and into multiple regions and environments with only changing two inputs to terraform apply. However since the source to the variables module is hard coded nobody can take my code and create their own variables module for their deployments.

[josephcaxton]

I need to be able to pass variable. Is Hashcorp looking to resolve this issue?

[GarinKartes]

With workarounds being provided and they intentionally made it this way, not likely we will see parameters in the source line.

I recommend using different folder paths and wiring up all relative pathing in your TF files.

From: josephcaxton notifications@github.com Reply-To: hashicorp/terraform reply@reply.github.com Date: Wednesday, December 5, 2018 at 6:30 AM To: hashicorp/terraform terraform@noreply.github.com Cc: Garin Kartes Garin.Kartes@alaskaair.com, Comment comment@noreply.github.com Subject: Re: [hashicorp/terraform] terraform get: can't use variable in module source parameter? (#1439)

I need to be able to pass variable. Is Hashcorp looking to resolve this issue?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fhashicorp%2Fterraform%2Fissues%2F1439%23issuecomment-444504173&data=02%7C01%7Cgarin.kartes%40alaskaair.com%7C1692108d43a74281574e08d65abe4217%7C0f44c5d442b045c2bf55d0fea8430d33%7C1%7C0%7C636796170540379315&sdata=44aW3hZTTeccEDntjYPI03TeU11tqXtlJSKfJThwknk%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FARwnyDDvgV-3yvBNCAQes2gsVqzbYiZNks5u19iXgaJpZM4D9Dyw&data=02%7C01%7Cgarin.kartes%40alaskaair.com%7C1692108d43a74281574e08d65abe4217%7C0f44c5d442b045c2bf55d0fea8430d33%7C1%7C0%7C636796170540389334&sdata=99pGIuhS1Td8MJQahoDjOJnsCWJGguO6x9amTi4BZco%3D&reserved=0.

[major0]

While I can understand the reasons for not supporting general var/local inclusion .. I feel that many (all?) of the above use cases could be resolved by adding${path.root} to the list of allowed local module source prefixes.

[damon-atkins]

Or even something like source yaml_lookup://../lookupfile.yaml which contains module name and source pairs. Or some sort of cli option --source_overrides=something.yaml The value is saved in the state, and warns if anything is different to the last run.

[jmturwy]

Another example as to why this is beneficial:

`####################### Global value ####################### locals { orgname = "acmeCorp" }

###################### module "iam" { source = "./iam/customer/${local.orgname}" org-name = "${local.orgname}" providers = { aws = "aws.customer-${local.orgname}" }

} ###################### module "s3-bucket" { source = "./s3/customer/${local.orgname}" org-name = "${local.orgname}" env = "production" providers = { aws = "customer-${local.orgname}" } } ####################### module "vpc" { source = "./vpc/customer/${local.orgname}" org-name = "${local.orgname}" env = "production" region = "us-westt-1" cidr-octs = "10.7" peer-account = "xxxxxxxxxxxxxx" peer-vpc = "vpc-xxxxxxxxxxxxxxxxx" peer-cidr = "192.10.0.0/16" }`

this would be called acmecorp.tf, we would just copy this module and renamed it to loonytoons.tf and change the local var to loonytoons thus saving a lot of copy pasta

[non7top]

Adding to a comment by richardgavel from Nov 14, 2018

Backend configuration is stored in .terraform/terraform.tfstate, so store module sources in there and require re-init if those change, i.e something like module.cluster1.app -> source="github.com/example/example"

Having such feature is particularly useful if you want to test new module version which is located in some feature branch in another (shared) repo, you then have to edit all paths to module manually and re-init anyways.

[BattleBrisket]

+1 for this. I have a git-based module to configure team permissions, and I have ~80 teams. That's a lot of wet, brittle code that won't stand up to any significant change in the repository structure.

If we cannot have the source set as a variable, could we specify some module-specific config values that would load at runtime? If I could store the git URL and a ref tag somewhere in tfvars, for example, that would meet my needs.

[anderson-dan-w]

A use I see easily popping up (in that literally my first project that I'm working on terraform with), I want to have multiple modules that I pull from, but I will always want those to use same branch, within a project:

prod_git_tag = "v.0.0.1"
staging_git_tag = "v.0.5.7"

and then

module "vpc" {
  source = "${git_repo_path}?ref=${git_tag}"
}

module "kubernetes-cluster" {
  source = "${git_repo_path}?ref=${git_tag}"
}
 # etc...

which seems pretty reasonable to me - when I pass in git_tag=prod_git_tag, now they all reference the same git_tag and can be updated with one line, rather than in all the various places.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.