[feat] add AGE Secrets Provider

cr1cr1 commented 1 year ago

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

Currently, for local secrets there is only passphrase provider, so a good addition would be able to use other encryption mechanisms like age
Would support multiple identities and eliminate the need to share passwords with multiple team members
Would help reduce the management of master secrets in an IaC repository (especially if it contains other types of configurations and secrets, not related to Pulumi)
Would help reuse existing secrets in the case of gitops repos that already use tools like sops+age to encrypt files that are stored in the same git repo, without depending on outside services for secrets
Age has a golang library https://pkg.go.dev/filippo.io/age and supports SSH keys
Example usage:
- Age public keys: --secrets-provider="age://age1mkllh99324xaywsh3uzr85pzwa05w7muhaga30denf5zmm88ufdsxqtyct,ssh-ed25519 AAAA...,ssh-rsa AAAA..."
- Age recipients file: --secrets-provider="age:///path/to/recipients.txt"
- To decrypt, secrets ("AGE-SECRET-KEY-1...", ssh private keys) must be provided from stdin or from filepath

I would like to try and implement this feature myself, as I believe is not very difficult to do.

Affected area/feature

area/secrets

PraveenNanda124 commented 1 year ago

Good issue

Frassle commented 1 year ago

What fortuitous timing! I was literally yesterday thinking about how we could move our secret providers to a one of our plugin interfaces. Both to help with some internal build dependencies (The actually pulumi engine executable wouldn't have to link directly to the aws sdk for example) but also to allow community members to contribute their own secret plugins.

We've got a hackweek coming up soon, I'll see if I can put a framework together for this then.

Frassle commented 1 year ago

So I did look into this for our hackweek. It looks quite doable, and I think I've already got a good idea of what the cloud and passphrase providers look like when pulled into plugins, which also defines what the grpc interface would look like. You can see these on my pluginSecrets branch, eg. https://github.com/pulumi/pulumi/tree/fraser/pluginSecrets/cloud-secrets.

There's a load of clean up we need to do through the main engine to enable this though, but I'm going to try and do that in small refactor passes over the coming weeks.

In the meantime it would help if you had a look at the proto definition for the secrets service: https://github.com/pulumi/pulumi/blob/fraser/pluginSecrets/proto/pulumi/secrets/secrets.proto

And comment here if you think it would be possible to implement an AGE provider against that interface. Happy to answer any questions you've got about the interface here on on community slack (@Fraser Waters, not @Fraser)

rafaribe commented 1 year ago

Is there any development on this? We use age/sops in our repos and this would be pretty awesome.

Frassle commented 1 year ago

I've been chipping away at it over the months to get in the refactors needed to make it more feasible to switch over to the plugin system, but this hasn't been priority work so it's been slow going.

I expect this will be possible at some point, but it's very hard to guess time scales for it right now.

omidraha commented 1 year ago

@rafaribe Do have an example of using Pulumi with SOPS or age?

folliehiyuki commented 1 year ago

Do have an example of using Pulumi with SOPS or age?

You can treat the Pulumi.<stack>.yaml file as an input to sops to encrypt every value there. The Pulumi secret backend can simply be local password with empty string, for example.