Options to store Terraform plan files with sensitive values

[hmrks]

Current Terraform Version

Terraform v1.0.9
on linux_amd64
+ provider registry.terraform.io/hashicorp/random v3.1.0

Use-cases

Using terraform plan -out in public cloud environments like Bitbucket pipelines or GitLab CI, without writing sensitive data in plain text to disk when creating Terraform plan files.

Attempted Solutions

Make it possible to store tfplan files on the remote backend, make it possible to encrypt them with a password/key, or make it possible to pipe it to a GPG command or similar, to avoid writing sensitive data to disk.

Proposal

Remote backends are an important feature in Terraform: they make it possible to use Terraform in a team - but they have advantages in security as well: they make it possible to store Terraform state files only on encrypted storage backends.

When running Terraform in automation, it is recommended to create Terraform plan files with terraform plan -out=<path> to ensure that only the very same steps specified by the plan flie will be performed.

Since Terraform plan files are just zip archives containing - among others - the entire state file, potentially sensitive data is written to disk. This can be a problem when running CI/CD pipelines in the public cloud, or when storing the working directory as build artifacts (as suggested by Hashicorp), if our CI/CD solution of choice does not provide encryption for them.

I think this could be improved by:

• allowing us to store Terraform plan files on a remote backend,
• adding an option to encrypt plan files with a password/passphrase/key,
• or making it possible to write plan files to stdout, then pipe them to a command which encrypts the contents before writing them to disk.

[apparentlymart]

Hi @hmrks! Thanks for sharing this use-case.

As you implied with your references to the automation guide, our current posture does rather assume that either you have a Terraform configuration that doesn't manage anything sensitive or that you're running Terraform in an isolated environment with a private filesystem and memory that other tenants can't access, which includes the saved plan file.

I think it could be interesting to prototype a variant of your last suggestion where there could be a way to explicitly instruct Terraform to "save" a plan file by starting a child process and writing into its stdin so it can potentially do encryption and/or send the file contents somewhere else, without Terraform being directly aware of either of those decisions. That seems more feasible than using Terraform's own stdout, because terraform plan already uses stdout for various other output.

Our current internal API for creating plan files takes a filename and tries to create the file itself, so we would need to make some minor adjustments to allow passing in an arbitrary writer too, so that the plan command could then deal with starting up this child process and passing its stdin (as an io.Writer) to the plan file writer. I think we'd also want to take care to ensure that the child process has some way to detect whether it received the entire file or only a partial file, in case we encounter some sort of error partway through writing.

I think we'd need to play with the design of this a little before deciding if it's a good path forward in practice, so we can use this issue to represent the use-case and hopefully find some time to prototype what I wrote here along with possibly other approaches, like the ones you suggested.

[hmrks]

Hi @apparentlymart!

Just wanted to add that a hacky workaround seems to exist for writing an encrypted planfile to disk, by forcing Terraform to write it directly to /dev/stdout. In this example I am using age:

terraform plan -out /dev/stdout | age -r $PUBLIC_KEY > tfplan.age

It does not work the other way around though, as Terraform expects a zip file:

age --decrypt -i $IDENTITY tfplan.age | terraform apply /dev/stdin

│ Error: Failed to load "/dev/stdin" as a plan file │ Error: zip: not a valid zip file

The same things happens when trying to read the decrypted planfile from a subprocess in Bash (terraform plan apply <(age --decrypt -i $IDENTITY tfplan.age)). When I decrypt this file in a separate step it works as expected.