Mount Objects/Sync Secrets By Regex Pattern From Key Vault

[rkapoor028]

Describe the solution you'd like [A clear and concise description of what you want to happen.]

Mount objects from Key Vault following a regex pattern. Could not find an existing way to do this.

Also can't find anyone asking this but it will be a useful feature if we can add Secrets using the regex pattern as well.

Is there a way to mount secrets, keys, certificate by specifying wildcards in the SecretProviderClass instead. It will help in case we need to mount many secrets from the Key Vault but all(secrets,keys,certs) follow a specific pattern eg:- frontend-msvc- , backend-msvc- etc. It will mount everything with that pattern in the volume from Key Vault.

Secrets by Regex Pattern

secretObjects:

• secretName: frontend-msvc-* type: kubernetes.io/tls data:
  • objectName: frontend-msvc-* key: tls.key
  • objectName: frontend-msvc-* key: tls.crt

Objects by Regex Pattern

objects:  |
  array:
    - |
      objectName: frontend-msvc-*
      objectAlias: frontend-msvc-* 
      objectType: secret  
      objectVersion: "" 

Azure App Configuration Supports something similar by loading Secrets from KeyVault, Configuration from App Configuration following a greedy mode.

https://docs.microsoft.com/en-us/aspnet/config-builder#mode

Would love to hear thoughts on this?

Environment:

• Secrets Store CSI Driver version: (use the image tag):mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.0.1
• Azure Key Vault provider version: (use the image tag):mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.10
• Kubernetes version: (use kubectl version): 1.19.X
• Cluster type: (e.g. AKS, aks-engine, etc): AKS

[aramase]

@rkapoor028 Thank you for opening the issue. This issue was initially created to get user consensus on this requirement: https://github.com/Azure/secrets-store-csi-driver-provider-azure/issues/61.

1. Adding this feature would mean the service principal/managed identity used for auth would require list access for secrets/keys/certificates in key vault. Today, we only require get permissions.
2. The regex if supported will only be for fetching the objects from key vault and writing it to the mount. For secretObjects to sync as K8s secrets it'll still require individual object names. A single k8s secret data field can't hold multiple values combined and the name being non deterministic is probably not a good approach.

[rkapoor028]

@aramase This would really be very useful, instead of defining repetitive objects bloating the SecretProviderClass file we can just match objects from KeyVault via a certain pattern.

For 2) Another approach can be that we define the name of the k8s secrectObject(msvc-frontend-secrets) that will be created/synced, and specify a pattern(msvc-frontend-*) based on which the Objects will be matched and loaded from the Keyvault as a Key/Value pair inside the secret. Keys inside the secretObject(msvc-frontend-secrets) can be set as the exact object name(msvc-frontend-secret1) in Keyvault so its pre-determined what the name of the key will be. For example

spec:
  provider: azure
  secretObjects:
  - secretName: msvc-frontend-secrets
    data:
      objectName: msvc-frontend-*
      objectType: Regex
    type: Opaque

Would sync the secret as below:-

kind: Secret
apiVersion: v1
metadata:
  name: msvc-frontend-secrets
  labels:
    secrets-store.csi.k8s.io/managed: 'true'
data:
  msvc-frontend-secret1: XXXXXXXXXXXXXXX
  msvc-frontend-secret2: XXXXXXXXXXXXXX
  msvc-frontend-secret3: XXXXXXXXXXXXXX
type: Opaque

Not sure, how much feasible it is.

[aramase]

For 2) Another approach can be that we define the name of the k8s secrectObject(msvc-frontend-secrets) that will be created/synced, and specify a pattern(msvc-frontend-*) based on which the Objects will be matched and loaded from the Keyvault as a Key/Value pair inside the secret. Keys inside the secretObject(msvc-frontend-secrets) can be set as the exact object name(msvc-frontend-secret1) in Keyvault so its pre-determined what the name of the key will be. For example

@rkapoor028 How are you consuming the content from Kubernetes secrets? Are you using the Kubernetes secret to configure as environment variables or actually reading the Kubernetes secret? I'm curious because if you use regex for secret data names, then the data keys aren't deterministic unless they're hardcoded in the yaml manifests for the env var/in the code.

[rkapoor028]

Yes, so in above approach, the secret data keys are the AKV secrets(msvc-frontend-secret1,msvc-frontend-secret2,msvc-frontend-secret3 so on..). We are referring those AKV secret name by the same name in the Deployment Yaml Manifests as the secret data key name(key: msvc-frontend-secret1, key: msvc-frontend-secret2, key: msvc-frontend-secret3 and so on...).

          env:          
          - name: BlobTableConnectionString1
            valueFrom:
              secretKeyRef:
                name: msvc-frontend-secrets
                key: msvc-frontend-secret1
          - name: BlobTableConnectionString2
            valueFrom:
              secretKeyRef:
                name: msvc-frontend-secrets
                key: msvc-frontend-secret2
          - name: BlobTableConnectionString3
            valueFrom:
              secretKeyRef:
                name: msvc-frontend-secrets
                key: msvc-frontend-secret3

So they are deterministic in the above approach. So the env variables BlobTableConnectionString1, BlobTableConnectionString2, BlobTableConnectionString3 etc. is what the app is expecting to be present and we are loading their values from k8s secret's appropriate data key.

[trhumphries]

This also might allow the drive to load all of the matching items via a single call to the provider where as today it has to perform a separate API call to get the value of each secret. Loading them one by one seems to be very resource intensive, where as the load used by the Configuration class Key vault extension loads them all in a few milliseconds.

[aramase]

This also might allow the drive to load all of the matching items via a single call to the provider where as today it has to perform a separate API call to get the value of each secret. Loading them one by one seems to be very resource intensive, where as the load used by the Configuration class Key vault extension loads them all in a few milliseconds.

@trhumphries AFAIK the list API in keyvault only providers the metadata about the secrets/certs/keys. It doesn't return the actual secret value. To get the secret value it has to be a GET call on the individual secret. With regex, the provider would still need to list all secrets, perform client side filtering to find secrets that match regex and make a GET call for each secret.

[trhumphries]

Well the AddAzureKeyVault extension to the Microsoft.Extensions.Configuration loads all of the secrets from a Key Vault several times faster than the CSI Driver does.