Template variables and bash substitution

[ejrgilbert]

Current Terraform Version

v0.14.7

Use-cases

Templating a large bash file is cumbersome in TF due to how templating variable syntax works. Currently, you must escape any bash variable including curly brackets (e.g. ${varname}), but not escape bash variable excluding curly brackets (e.g. $varname).

For large bash files, this gets tricky and is hard to debug since you don't have a visual syntax difference between the bash variable references and the TF variable references.

Attempted Solutions

Technically there is a workaround, using the $$ to escape bash vars including curly brackets...it's just really annoying and would be better to have different syntax.

Proposal

Change syntax to be similar to how Ansible does it: {{ terraform_variable }}. This wouldn't conflict with bash naming. Maybe to move forward make this an optional syntax? So there would be backward compatibility?

References

Bringing a comment from a few years ago back to light from an old closed issue.

[tidzo]

templatefile() should have an additional optional argument varstyle, or even separate variable_start_string and variable_end_string options like ansible's template module.

[colemickens]

How is this an issue, and how is there not any sort of documented workaround, or note, or anything. Not even a note on how to escape it? I'm sort of shocked, what are people with well-written bash scripts doing, exactly?

I have to agree with a comment from that thread, this gap is a bit glaring.

[krowlandson]

I was going to raise a similar issue for different reasoning, but will add my scenario to this one to prevent duplication.

In our case, we have a library of JSON templates which we want to be able to use across multiple code bases. To make them more universal, we would really benefit from a more common templating format.

As suggested by @ejrgilbert, allowing support for the more common {{ terraform_variable }} format would provide greater compatibility with other tools such as Ansible, jinja2, etc.

As a stretch goal, it would also be good to:

1. Provide the ability to control which templating format is being used; whether by introducing a new optional input to the existing function, or by creating a new template file function.
2. Provide support for additional template capabilities; for example, supporting control structures such as if and for to provide more powerful templating capabilities.

[apparentlymart]

As a side-project (i.e. not an official HashiCorp project) I implemented apparentlymart/bash, which is a Terraform provider offering a function that's intended to be a different take on this issue: instead of using template interpolation to paste strings into a bash script, instead use bash's own variable-related features directly so that Terraform's template language isn't in the mix at all.

Using that approach also neatly avoids many of the typical concerns about properly escaping data values so that they don't get misinterpreted as commands to run, or cause syntax errors when certain characters appear. I tried to give good tips in the documentation for how to use it effectively to write robust scripts that will work with any variable values.

Since this was my personal project and not supported by HashiCorp I don't intend this comment to represent a solution to this issue, but I'm mentioning it here because it addresses the same use-case in a different way and so might be useful to anyone who is frustrated with trying to generate code in one language using code in another language that happens to use similar syntax.

[apparentlymart]

Now that Terraform supports providers contributing additional functions to the Terraform language (which my previously-mentioned provider is exploiting), it's also possible for someone to write a provider that wraps some other template implementation, if you think that pasting strings together is still the better approach but you'd rather use someone else's template syntax to do it.