Multiple backends for a secret

[vitiho]

It would be nice to have ability for multiple backends for the same secret, for example:

gcloud_privatekey: ?{kapitan:namespace/stage/gcloud_privatekey|base64}

and under the secrets definition have:

secrets:
  kapitan:
     gkms:
         key: 'projects/<project>/locations/<location>/keyRings/<keyRing>/cryptoKeys/<key>'
     gpg:
       recipients:
          - name: example@kapitan.dev
           fingerprint: D9234C61F58BEB3ED8552A57E28DC07A3CBFAE7C

That way it would first try gkms, and if service is down, it would fallback to gpg... (Ideally priority defined by which ones comes first, but since dictionary is not sorted maybe having priority: field could be an option.

[MatteoVoges]

Hey @vitiho , in your description you said, that this would work as fallback if a service is down. I don't see much value in this, because the initial problem has nothing to do with kapitan, rather with the secret service you are using.

When I first read the title, I thought of a way to specify two secret engines for the same secret in the ref tag itself, to put the secret/ref in both backends ... This would solve your problem and provide a cool feature to simplify your inventory a bit.

Do you have any updates or thoughts on this?

[vitiho]

Well if we could specify two secret engines for the same secret that would work for the fallback as well. Main value in that is: In case of DR, you can have all your manifests in Kapitan, but if your secrets are not available you'd loose ability to deploy (most likely you'd need to recreate a lot of them, but there could be some which would be useful to keep).

[MatteoVoges]

What do you think about the following process: The ref-tag ?{gkms,base64,vaultkv:path/to/ref||random:str} would look up the secret in order from left to right. If the first backend is reachable, we will contain the current behavior, so if the secret exists locally, we will try to get it from the backend. If not, we will generate it and try to store it in the backend.

Now we have to strategies to deal with the other specified backends:

• logical OR: this means that we try the other backends only if the first one (or the previous one) couldn't store the secret and is unreachable. Then we will go from left to right through the list of backends and try them one after another until we succeed. The Problem would be, that if the first backend worked in the first place, but later it doesn't work anymore, then backend2 in the list wouldn't have the secret...
• logical AND: The stated problem from option 1 wouldn't be the case here, because the secret gets stored in each backend (if its reachable at the time). However this method can lead to inconsitencies between secrets, especially if you edit your secrets manually. I would then prioritize the first reachable backend in the list.

Which strategy do you prefer?

What also has to be mentioned is that the backends store different attributes like fingerprints or vault_params and we have to take care of them.

[vitiho]

Left to right sounds good, I'd probably do logical OR with the error message if remote backend is unavailable. Realistically you'll write the secrets once in a while (depending on your rotation policy), but read them on every deploy.

[MatteoVoges]

But then if the first backend is unavailable and you wrote your secret just in this one (because of the logical OR) the other backends don't have it and you can't deploy... Or am I missing something?

[vitiho]

Well, right, but you'd know that it only wrote into one backend... cause it would complain at the time of the writing. So for example: you use vault as primary and gpg as the next one... if vault is down then you know during the write the it didn't work and you'll need to try again. (Maybe AND would work better in this case for writes). For the read if vault is not available it'll look the next one which would be gpg and you'd still be able to deploy apps.

[MatteoVoges]

So you would write your secret to all of the (available) backends and when reading you'd do the strategy from left to right, which backend is the first that has the secret. Alright I think I'm going to implement it. Stay tuned!

[github-actions[bot]]

This issue is stale because it has been open for 1 year with no activity. Remove the stale label or comment if this issue is still relevant for you. If not, please close it yourself.