Templating requires variables to be set that are not reached by if blocks

[tpdownes]

Terraform Version

Terraform v0.12.13
+ provider.aws v2.30.0
+ provider.random v2.2.1

Terraform Configuration Files

  provisioner "file" {
    content = templatefile("${path.module}/${var.environment_name}_install.sh.tmpl",
      {
        flavor = "aws",
        region = "${data.aws_region.current}",
        controller = "demo-controller",
        ami_id = "ami-000",
        subnet_id = "${random_shuffle.subnet_id.result[0]}",
        security_group_id = "sg-000",
        key_pair = "tpdownes-macbook-pro",
        image_user = "centos"
      })
    destination = "/tmp/install.sh"
  }

The relevant (breaking) section of the template:

%{ if flavor == "gce" }
PROJECT=${project}
...
%{ endif }

Debug Output

╰─>$ terraform console                                                                                                                                               (base)
> templatefile("${path.module}/demo_install.sh.tmpl", { flavor = "aws", region = "us-west-2", controller = "navops-demo-controller", ami_id = "ami-01ed306a12b7d1c96", subnet_id = "subnet-001", security_group_id = "sg-0733058924ed3b13f", key_pair = "downes-macbook-pro", image_user = "centos" })

>
Error: Invalid function argument

  on <console-input> line 1:
  (source code not available)

Invalid value for "vars" parameter: vars map does not contain key "project",
referenced at
/Users/tdownes/repos/test-install/demo_install.sh.tmpl:17,11-18.

Expected Behavior

I think it's reasonable to expect the templating engine not to care about the existence of a variable set inside an if block that won't be reached.

Actual Behavior

The templating engine, in fact, borks when a non-existing variable is referenced inside an if block.

Steps to Reproduce

Create file test.tmpl

%{ if env == "test" }
${test_only}
%{ else }
${env}
%{ endif }

Run terraform console

> templatefile("${path.module}/test.tmpl", { env = "not_test" } )

>
Error: Invalid function argument

  on <console-input> line 1:
  (source code not available)

Invalid value for "vars" parameter: vars map does not contain key "test_only",
referenced at /Users/tdownes/test.tmpl:2,3-12.

>

> templatefile("${path.module}/test.tmpl", { env = "not_test", test_only = "Mamma Mia" } )

not_test

[yves-vogl]

Have you been able to workaround this issue? I'm dealing with this symptom for template to create dynamic AWS Lambda@Edge functions inside a module.

[tpdownes]

I just set every variable in a template to the empty string if I'm not going to use it.

[tpdownes]

I'll spell out my use case. I have a user-data / startup script for setting up an application after an instance is provisioned. The details of that script depend on the cloud provider in use (AWS/GCP/Azure). So I have a bunch of blocks that are basically "if AWS do X" and "if GCP do Y" even though the rest of the script is common for the application regardless of the provider. The provider-specific stuff happens at several points in the middle of the script which makes it hard to break apart.

It's "nicer" to use Terraform templating because the resulting script has the code for only the provider in question and no bash if/else blocks for other providers.

[jpmontez]

I'm running into this issue, too. The issue's expected behavior makes sense to me, too. My workaround was setting a variable for each potential outcome to make it more "explicit". I suppose setting it to an empty string would make sense, too.

[rubiagatra]

I am facing this issue also

[abdullahkhawer]

There are 2 solutions:

1. Escape that variable in shell script using an extra $ before it. For example, PROJECT=$${project}
2. Add that variable as project = "$project" under the list of variables for file in terraform script and it will replace ${project} with $project and the script will still work.

Further details: 2nd solution would help as well as it will set its value to $project and then $project will pick the value when if condition will be true as that part of the code is actually setting the value. 1st solution will solve the issue and is the preferred solution but sometimes you cannot escape it using $ and you have to use 2nd solution which is actually replacing it with the code instead of value. It is more like a hack. I used it in while read line to escape $line which was enclosed with escaped double quotes and $$ didn't work and Terraform still threw error upon apply.

[tpdownes]

Escape that variable using an extra $ before it. For example, PROJECT=$${project}

You've misunderstood the issue. It's not BASH variable requiring escaping, it's a Terraform variable that is only used when that if statement evaluates to true. And yet, that variable is required even when the if evaluates to false. I believe that's non-intuitive behavior.

[abdullahkhawer]

@tpdownes I have updated my comment to add one more solution. I do agree with you and it would be better if handled by Terraform but the 1st or 2nd solution can be used as a workaround for the time being as even if the part of the code under if doesn't run everytime, Terraform throws this error and we need to make Terraform work irrespective if that variable comes under if or not. I faced the same issue when I used while read line and wasn't able to escape $line which was under if condition.

[dataviruset]

I'm trying to build a generic template for many of my service definition files, and some services don't have certain variables, but I don't want to define variables and set them to empty strings for the services without those variables. I tried using if can(myvariable) and even if contains(keys(vars), "myvariable"), but without success.

Error messages:

Invalid value for "vars" parameter: vars map does not contain key "myvariable",

and

Invalid value for "vars" parameter: vars map does not contain key "vars",

[sanbotto]

This is still an issue with Terraform v1.4.6. It would be really convenient to be able to do something like this inside templates:

${ coalesce( some_var, "some_default_value" )}

That way, you'd only need to set some_var when using a non-default value.

[matteoMiglio]

Yes, same problem here. I'm creating a generic template file with a for loop but I don't want to set an empty value for the federation_targets variable where it's not used.

Terraform file with the variable set

    templatefile("template.tftpl", {
      host = "prometheus-blackbox-exporter.monitoring.prod.local"
      federation_targets = [
        {
          name   = "federate_api_endpoint"
          module = "http_2xx_api_intranet"
          host   = "prometheus.common.local"
        }
      ]
    })

Terraform file with variable not used

    templatefile("template.tftpl", {
      host = "prometheus-blackbox-exporter.monitoring.dev.local"
    })

Template file (portion of the file)

%{ for target in federation_targets }
   - job_name: '${target.name}'
     scrape_interval: 15s
     honor_labels: true
     metrics_path: '/federate'
     params:
       'match[]':
         - '{job="${target.module}-prometheus-blackbox-exporter"}'
     static_configs:
       - targets:
         - '${target.host}:9090'
%{ endfor ~}

Error

Invalid value for "vars" parameter: vars map does not contain key "federation_targets", referenced at ...

[verbedr]

So what I did was adding a variable "optional" which is an object that contains all option values. Then you can do try(optional.fedaration_targets, []) or ${coalesce(optional.some_var, "some_default_value")}.

It's not that elegant, but at least it worked, and the impact was minimal in my use case. Also, the "optional" can be anything as long as the root element is defined it will work.