search

cloudposse / atmos

👽 Terraform Orchestration Tool for DevOps. Keep environment configuration DRY with hierarchical imports of configurations, inheritance, and WAY more. Native support for Terraform and Helmfile.
https://atmos.tools
Apache License 2.0
773 stars 95 forks source link

SOPS support #592

Open kevcube opened 6 months ago

kevcube commented 6 months ago

Describe the Feature

It would be great if Atmos had some interface for retrieving secrets from SOPS to be passed into terraform configuration.

Because we do our terraform configuration in Atmos' YAML, our TF configuration remains the same across environments/deployments. This is usually a good thing, but when something like a secret would change across environments, now we either need our terraform configuration to have some dynamic portion where it will grab the correct secret, or we would need to put the secret in raw YAML which isn't gonna happen.

I'm not sure about the extensibility of YAML

Expected Behavior

Some function in Atmos YAML to decode a SOPS secret from a given path.

Or some function in Atmos to read our SOPS files based on which stack/environment we are currently in and make those secrets available to reference in the YAML somehow.

Use Case

Secrets management can be better integrated with Atmos.

Describe Ideal Solution

See expected behavior

Alternatives Considered

No response

Additional Context

Terraform has a SOPS provider that we can use to do this work in our modules, and Terragrunt has native SOPS functions that can be used to do this in orchestration. Their implementation can be a reference.

osterman commented 6 months ago

We've recently extended atmos to support all types of datasources, although most are from gomplate. Unfortunately gomplate doesn't provide native SOPS.

It's a really interesting idea if stack configs could reference a datasource for SOPS encrypted secrets, guarded by cloud-specific keys like KMS.

See how we configure data sources today: https://atmos.tools/cli/configuration#templates

All we would need to do is add a datasource for a sops-encrypted file, then we could support key/value lookups using all the normal mechanisms.

https://github.com/getsops/sops

Gowiem commented 6 months ago

Love this idea -- Good issue @kevcube as this would definitely help us since we love SOPS so much.

@osterman is it possible to create datasources that are just for atmos? Or do they need to come from the upstream gomplate project?

osterman commented 6 months ago

We can include them simply in the Atmos core, without requiring them in the upstream dependencies. Also, if any of your team would be interested in getting more involved hands-on, let me know and we can coordinate.

zadm commented 5 months ago

This will be a very interesting feature to add to atmos

Currently, I am using terragrunt which has a natif support if sops datasource and this block me to move into atmos