Populate a single kubernetes secret from multiple Vault secrets

[fer1592]

Is your feature request related to a problem? Please describe. Currently, for VaultStaticSecrets we can only specify a single secret in Vault as the source, which forces us to, weather create and mount many k8s secrets when we have those distributed in many Vault secrets, or to create secrets in Vault with a lot of key/values, that might be repeated across many more Vault secrets.

Also, there is another use case. We would like to remove read permissions to (human) users, and allow them to only create/update/delete secrets in the KV engines. However, by doing that, the UI won't show which are the existing keys in a secret when creating a new version, forcing users to re-enter each key/value for the new secret version. This could be solved by the new feature KV patch+subkey, however that seems to be just for the Enterprise edition, not the community one. So, another solution for this problem would be to allow VaultStaticSecrets to get data from multiple paths, and instruct our users to create secrets in Vault with a single key/value, where the key is a known word, or same as the secret name.

Describe the solution you'd like It would be great if we could populate a single k8s secret with the values from different secrets in Vault.

Describe alternatives you've considered The External Secrets project allows to specify different paths on the ExternalSecret for the vault provider. Eg:

---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: example-store
  namespace: example
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      path: "kv-v2-example-secrets"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "kubernetes-service-account"
          serviceAccountRef:
            name: "example"
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: example-external-secret
  namespace: example
spec:
  refreshInterval: "60s"
  secretStoreRef:
    name: example-store
    kind: SecretStore
  target:
    name: example-sync
  data:
  - secretKey: keyforsecret1
    remoteRef:
      key: testing/secret1
      property: secret1
  - secretKey: keyforsecret2
    remoteRef:
      key: testing/secret2
      property: secret2
# It creates the following secret
# apiVersion: v1
# kind: Secret
# metadata:
#   ...
#   name: example-sync
#   namespace: example
# data:
#   keyforsecret1: <b64 encoded value of testing/secret1 secret1>
#   keyforsecret2: <b64 encoded value of testing/secret2 secret2>
# immutable: false
# type: Opaque

Additional context Although it's possible to use External Secrets, we would really like to have a similar feature in VSO, since not only External Secrets doesn't support DynamicSecrets, but also it will be one more tool to maintain.

Thanks in advance!

[hamishforbes]

Yeah :+1: for this from me, there's this closed issue that's actually asking for this: https://github.com/hashicorp/vault-secrets-operator/issues/118

I've been pondering solutions to this problem

a) you just make spec.path -> spec.paths as an array of keys to fetch. The Secrets object in templates could then become a map keyed by path? This solves the problem for static and dynamic secrets separately, but doesn't let you combine dynamic and static into a single k8s Secret.

b) Decouple the templating / (k8s) secret creation from the (vault) secret fetching, renewing etc Introduce a new CRD that can reference VaultStaticSecret and VaultDynamicSecret resources (and SecretTransformations) to define the destination (k8s) Secret and templates. When a VaultStaticSecret or VaultDynamicSecret reconciles it can then trigger any linked template to also reconcile and update the (k8s) Secret

e.g.

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  name: foo
spec:
  vaultAuthRef: default
  mount: kvv2
  type: kv-v2
  path: foo
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  name: bar
spec:
  vaultAuthRef: default
  mount: kvv2
  type: kv-v2
  path: bar
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
  name: dynamic
spec:
  vaultAuthRef: default
  mount: db
  path: creds/my-db-role
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultSecretTemplate (or whatever)
metadata:
  name: myapp
spec:
  secretRefs:
   - name: foo
   - name: bar
   - name: dynamic 
  rolloutRestartTargets:
   - kind: Deployment
     name: myapp
  destination:
    create: true
    name: myapp-unified
    transformation:
      excludeRaw: true
      excludes:
      - .*
    templates:
      myapp-config:
        text: |
          db.username: {{ .Secrets.dynamic.username }}
          db.password: {{ .Secrets.dynamic.password }}
          config.foo1: {{ .Secrets.foo.field1 }}
          config.foo2: {{ .Secrets.foo.field2 } }
          config.bar: {{ .Secrets.bar.field1 }}

[fer1592]

My $0,02, both options would actually solve our use case, however option b looks a lot more flexible (it might become very useful in the future). Thanks for taking a look at it!