search

pulumi / esc

Pulumi ESC (Environments, Secrets, and Configuration) for cloud applications and infrastructure.
https://www.pulumi.com/product/esc/
Apache License 2.0
216 stars 7 forks source link

Support for "syncing" or "pushing" to external systems #58

Open lukehoban opened 11 months ago

lukehoban commented 11 months ago

While the esc CLI and REST APIs can be used today to pull configuration down into many existing places where environments will be consumed, there are also many systems that store their configuration in more “walled gardens” which would need to be pushed into for optimal usability.

For example, pushing configuration values into a CI/CD system’s configuration system to avoid needing to copy/paste configuration and secrets manually into their UI.

We want ESC to offer a “push”/”sync” option that can be configured to inject configuration from an environment into such systems directly, so that you can still benefit from the usability of their native configuration systems, without having to take on the problems of configuration sprawl, long-lived static secrets and duplication and copy/paste of secrets.

We anticipate this supporting things like GitHub Actions env secrets, and many other similar use cases.

EronWright commented 11 months ago

A good use case may be to sync into Kubernetes Secrets and ConfigMaps. Maybe the Pulumi Kubernetes Operator could implement this by providing an API. e.g.

kind: SecretSource
metadata:
  namespace: shopping
spec:
  environment: shopping-service-staging
  secretName: shopping-service
EronWright commented 11 months ago

Another idea: sync with GitHub Environments, e.g. by pushing secrets. A benefit is to link ESC environments with protection rules, as described here. For example, allowing the release/* branches to access a particular ESC environment.

gunta commented 8 months ago

Looking forward to this.

At least we can expect to sync ESC secrets to GitHub Actions in a secure way.

MMartyn commented 7 months ago

Was going to suggest adding this to ESO but I see Engin beat me to it already :)

dirien commented 7 months ago

@MMartyn the PR (https://github.com/external-secrets/external-secrets/pull/2997) is merged!

I do not know their release cycle but expect it soon to be GA! 😍

cleverguy25 commented 2 months ago

Added to epic https://github.com/pulumi/pulumi-service/issues/18145

komalali commented 1 week ago

I wrote up a blog post about our recommended pattern for syncing to external systems - with links to examples on how to do this for AWS Secrets Manager, AWS Parameter Store, Azure Key Vault, GCP Secrets Manager, GitHub Secrets and HashiCorp Vault. Would love for y'all to test it out and let us know what you think!

https://www.pulumi.com/blog/esc-sync-with-iac/