External program backend

[ncmans]

Current Terraform Version

v0.12.16

Use-cases

Create an escape hatch for when an unsupported state backend is desired by the user.

Some scenarios:

• The long standing issue with plain text state files stored locally on disk. In presence of support for call to external programs to store and retrieve state, the user can simply call GPG to encrypt and decrypt the state before passing it between terraform and the underlying storage.
• User wants to store state in an unsupported storage like in kubernetes as a resource, in a windows share drive, as a file on dropbox, in a custom key value store, encrypted in a git repository with versioning, etc.
• User wants to validate the state to ensure it conforms to certain security or other policies before accepting it.

Attempted Solutions

Currently, the only available and customizable backend is the http backend. However, it involves running a program separately first and handling of the HTTP protocol. This adds quite a bit of friction compared to say a one line shell script that pipes the state into kubectl to be stored on kubernetes. The identity of the OS user running terraform is also lost which makes running this setup securely a little challenging.

Proposal

I propose to having a backend called exec which like the http backend allows defining custom actions for each state event. The actions would be the shell command that will be run by terraform and state passed onto them in stdin and read from stdout of the called program.

References

[pkolyvas]

Hey @ncmans thanks for the suggestion. :)

We're currently hard at work thinking about how to open up more options for state backends without putting the Terraform team in the critical path of merging, maintenance and ownership.

We don't have an answer yet, but appreciate the effort you've put in here and hope to return in the not-so-distant future with some solutions specifically around when an unsupported state backend is desired by the user.

Stay tuned!