vault sync to google secret manager naming convention

[nia-potato]

Is your feature request related to a problem? Please describe. when using the new vault sync feature to Google Secret Manager, every secret that is synced to GSM seems to have prepended vault accessor before the secret name, is there any way to not prepend the accessor but just use the secret name?

Describe the solution you'd like when using secret sync feature, instead of the secret name being vault_kv_XXXXXX_secret_name can we just have secret_name

Describe alternatives you've considered toggle somewhere to switch this behavior on or off?

Explain any additional use-cases when replacing existing secrets in google secret manager, we will need to make bulk code changes to change name all over the place instead of just using the previous secret name after sync.

[nia-potato]

also it seems like if the secret in vault has a very long path such as path/to/secret/in/vault then the name of the secret in google secret manager will become vault_kv_XXXXXX_path/to/secret/in/vault_secret_name. Is there anyway currently we can bypass this forced naming convention on the destination secrets for the secret sync option?

@zofskeez

[maxcoulombe]

Hi @nia-potato thanks for that request and the level of details.

It is now possible to specify your preferred name pattern using a secret_name_template attribute. The default pattern does not change and is still vault/{{ .MountAccessor }}/{{ .SecretPath }} but it is no longer hardcoded.

Starting with version 1.16-rc, every destination type allows you to customize this name pattern by configuring a secret_name_template field to best suit your use case. The templates use a subset of the go-template syntax for extra flexibility.

The following placeholders are available:

Placeholder	Description
DestinationType	The type of the destination, e.g. "aws-sm"
DestinationName	The name of the destination
NamespacePath	The full namespace path where the secret being synced is located
NamespaceBaseName	The segment following the last / character from the full path
NamespaceID	The internal unique ID identifying the namespace, e.g. RQegM
MountPath	The full mount path where the secret being synced is located
MountBaseName	The segment following the last / character from the full path
MountAccessor	The internal unique ID identifying the mount, e.g. kv_1234
SecretPath	The full secret path
SecretBaseName	The segment following the last / character from the full path
SecretKey	The individual secret key being synced, only available if the destination uses the secret-key granularity

Let's assume we want to sync the following secret:

$ VAULT_NAMESPACE=ns1/ns2 vault kv get -mount=path/to/kv1 path/to/secret1 ========== Secret Path ========== path/to/kv1/data/path/to/secret1 ======= Metadata ======= (...) === Data === Key Value --- ----- foo bar

Let's look at some name template examples and the resulting secret name at the sync destination.

Name template	Result
prefix-{{ .SecretPath }}	prefix-path/to/secret1
{{ .SecretBaseName | uppercase }}	SECRET1
{{ .MountAccessor }}_{{ .SecretKey }}	kv_1234_foo
{{ .SecretPath | replace \"/\" \"_\" }}	path_to_secret1

Name templates can be updated. The new template is only effective for new secrets associated with the destination and does not affect the secrets synced with the previous template. It is possible to update an association to force a recreate operation. The secret synced with the old template will be deleted and a new secret using the new template version will be synced.

So coming back to your original demand if you set: secret_name_template = "{{ .SecretBaseName }}" this should give you the secret_name end result with your example.

We hope this solution works for you and is flexible enough for others as well!

[nia-potato]

@maxcoulombe thanks for the detailed response, i gave it try on rc1 and was getting this error, did i do something wrong?

vault write -f sys/sync/destinations/gcp-sm/test project_id=test granularity=secret-key secret_name_template="{{ .SecretBaseName }}"  
Error writing data to sys/sync/destinations/gcp-sm/epos-public: Error making API request.

URL: PUT https://vault.hostname.com/v1/sys/sync/destinations/gcp-sm/test
Code: 400. Errors:

* secret name template is invalid: external name template should contain reference to the secret key

granularity seems to go thru fine, its just the secret_name_template

i also tried doing this directly on the UI level, sometimes i will get permissions denied with a admin token.

i cannot test this on rc2 since the GCP workload identity code that works with secret sync is not included.

[maxcoulombe]

Hey @nia-potato , to fix this error you only need to slightly modify the name template. I think secret_name_template="{{ .SecretKey }}" would give you the result you are looking for.

There's a bit more info on the doc that will be released with 1.16. Here is a preview.

When a destination's granularity is set to secret-key Vault verifies if the name template contains at a minimum the placeholder {{ .SecretKey }} to avoid unintentional name collisions.

If we take a simple example with 2 keys:

$ VAULT_NAMESPACE=ns1/ns2 vault kv get -mount=path/to/kv1 path/to/secret1

========== Secret Path ==========
path/to/kv1/data/path/to/secret1

======= Metadata =======
(...)

=== Data ===
Key    Value
---    -----
foo    bar
baz    buzz

If Vault allowed the {{ .SecretBaeName }} template, both keys would be synced under the name secret1 and overwrite one another. Verifying there is at least the {{ .SecretKey }} placeholder forces foo & baz to be part of the name to minimize the odds of collisions. The sync feature uses a last-write-wins strategy and without some form of unique ID we are never 100% safe from name collisions but this check is a best-effort to prevent a potential pitfall.

Hope that make sense!

I'm surprised that the GCP workload identity code is not in RC2, let me reach out to the engineer who worked on this to double-check check this is not a mistake.

[nia-potato]

i see, i will just need to a add a automation that automatically writes the SecretBaseName as the SecretKey then use the secret_name_template="{{ .SecretKey }}" so that gets the same name and overwrites the destination existing secret within google secret manager.

Thanks

for the rc2 issue, i think @austingebauer is aware already.