search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.42k stars 9.5k forks source link

Alternative template escaping syntax that doesn't clash with POSIX shells #27895

Open techdragon opened 3 years ago

techdragon commented 3 years ago

Current Terraform Version

Terraform v0.13.5

and

Terraform v0.14.7

Use-cases

When building things like cloud-init / user_data boot scripts for bringing a new machine online with a desired configuration, its entirely normal to have such scripts pull data from their runtime environment to discover things that cannot be known or populated by terraform ahead of time. For instance AWS instance/volume IDs, dynamic IP addresses that have yet to be allocated, and other things like these in almost all cloud infrastructure providers.

Using $${} to escape shell script variables when these are used with the templatefile() function makes them harder to work with, critically it makes it more difficult to run static analysis and security audits against these script files to ensure we aren't introducing vulnerabilities in them as the tools cannot parse the files anymore. No matter how we cut the problem, we're going to just shift the risk from "this is wrong in terraform" to "(we made a mistake|have a bug with) converting the source into what terraform will render this into on a real machine" Unless terraform can use an alternative escape sequence that will play nice with other tools that have to analyse the templates.

Also... it shouldn't be expected that any of the terraform analysis tool can perform such template content checks. tfsec cant be expected to do the job of shellcheck, thats not a fair expectation to place on the tfsec developers.

Attempted Solutions

Using the existing $${ escape sequences. Carefully constructing the shell scripts to minimise the number of "ignore this" statements and breaking shell scripts into smaller parts so that the parts that actually use terraform templating are as small as possible to allow the checking of other parts, but this doesn't really work well with user_data/cloud-init which likes having one big script to run at boot.

Proposal

The best option would be configurable escape sequences, but failing that, one that was better at playing nice with shell scripts (and other things that use ${} for variables) would be necessary to solve this issue.

References

wwalker commented 3 years ago

Please! We use ShellCheck on our shell scripts, and the terraform template language confuses * Me * and ShellCheck.