Easier way to render/debug templates

[zapman449]

Terraform Version

Terraform v0.10.6

Debugging Terraform templates is far more time-consuming than it should be. There should be a simple way to "Render this template for me, so I can debug it further". AWS complicates this greatly by primarily using Bash scripts as user-data, and Terraform Templates and Bash both use the ${} concept extensively.

Attempted work arounds:

1. echo "data.template_file.TEMPLATE.rendered" | terraform console : Works great IFF you've run terraform plan ; terraform apply before hand. This slows down the testing process

2. Follow this method: https://stackoverflow.com/questions/37887888/print-terraform-template-rendered-output-in-terminal . (Boiled down: run terraform refresh then terraform console) : This works as well, but in a large codebase, terraform refresh can take a long time.

3. This is a third method out there: http://blog.aurynn.com/2017/2/23-fun-with-terraform-template-rendering : It works, but there are two major flaws. 1) It requires building an 'out of codebase' harness to run it, which is hard in complex situations. 2) If you're building a shell script, the HEREDOC syntax can trigger interpolation of things in the invocation. Also, until you light upon the $$() syntax for "sub-shell inside terraform template" it can take several minutes to render while it fails to run the command.

What I desire: "A simple and fast way to render Terraform Templates". Ideally in a single command. It could be something like terraform plan -template-only -output-dir=/tmp and name all the templates after their data "template_file" "<NAME>" name.

[apparentlymart]

Hi @zapman449! Thanks for sharing this use-case.

Currently the template rendering in Terraform is just a provider like any other, and so this sort of special handling is not really possible. One approach we could take here is to make templates a more integrated part of Terraform, though in the short term we will not make this change because we're currently engaged in work with improving the configuration language and template language and so such refactoring would conflict with that work. We can revisit that idea once the current refactoring is further along.

Thinking about solutions that would work with the current architecture, to put your request in Terraform's own internal terms what would need to happen is to refresh the data-source in-memory, just like a plan would do, and then print out the result of some expression. This would essentially be your option 2 but promoted to an "official" command and with the advantage that it would not actually update the real state, as terraform refresh does.

I'm not sure what would be a good name for such a command, but let's call it query-data for the sake of example:

# hypothetical example --- not actually implemented
$ terraform query-data data.template_file.foo.rendered

This would be different than using terraform console because it would first refresh data.template_file.foo in-memory before then printing out the value of the full expression, which would in this case be the rendered template. This would then generalize to any attribute of any data source, as long as it can be serialized as a string to print it.

---

Looking at this from a different angle, also seems worth mentioning that we usually recommend keeping the template portion small when generating scripts, for the reasons you mention around the complexities of keeping all the different levels of escaping in order.

userdata.sh:

# This script expects certain environment variables to be set already when it's run,
# but otherwise it's just a normal shell script.

echo $SOMETHING_DYNAMIC_FROM_TERRAFORM

userdata.tmpl:

#!/usr/bin/env bash

SOMETHING_DYNAMIC_FROM_TERRAFORM=${value}

${script}

In the terraform config:

data "template_file" "example" {
  template = "${file("userdata.tmpl)}"

  vars = {
    value = "hello world"
    script = "${file("userdata.sh")}"
  }
}

This means that the bulk of your script code is just a static file that you can write in your editor's native shell script mode (if it has one) and not worry about any weird escaping. It then gets prepended with a small amount of template script that deals with setting the variables. The script can then also be run directly from a shell prompt for testing, by setting the environment variables manually:

$ SOMETHING_DYNAMIC_FROM_TERRAFORM="yo" bash userdata.sh

This approach doesn't suit all situations of course, but where it can work I'd recommend it just to reduce the friction in writing and testing the script.

[JustinGrote]

I made this workaround that also works pretty well: https://github.com/hashicorp/terraform/issues/18242#issuecomment-446020085