Kubernetes Config extension richer injection

[lordofthejars]

Description

Currently Kubernetes Config extension allows you to inject multiple secrets by doing quarkus.kubernetes-config.secrets=my-connect,my-cluster-cluster-ca-cert. That's fine, but if I am correct, what's is doing is putting all keys in the same map, and then you refer to them as values in each key:

kafka.ssl.truststore.password=${ca.password}

But what happens if there is a key with the same name in both secrets?

I think it will be nice to have something like Strimzi project to inject secrets/configmaps adding a richer expression language:

${secrets:debezium-example/debezium-secret:password}

The format is namespace/secretName:key. This gives you more flexibility, and you know exactly that:

1. The property comes from a secret (in the current case, you have the ${key}, and this property could come from a secret or not.
2. You know exactly the namespace and the secret the value is coming

Apart from that, there is one use case that would be nice to cover; let me show an example:

kafka.ssl.truststore.location=/tmp/cluster/ca.p12
kafka.ssl.truststore.password=${ca.password}

TrustStore needs to be a file; the thing is that the content of this file is created as a Kubernetes Secret, so we've got a Kubernetes Secret with the certificate authority in one key (ca.p12) and the password in another key (ca.password). The problem is since it's a location, we need to materialize the content into a file, so we need to create a secret volume and so on.

So, after all, we are mixing two strategies, injecting secrets from a volume and injecting using the Kube API. It would be great if we could do something like:

kafka.ssl.truststore.location=${materializedsecret:debezium-example:my-cluster:ca.p12}
kafka.ssl.truststore.password=${secrets:debezium-example:my-cluster:ca.password}

Where materializedSecret takes the key, creates a file in the filesystem, and injects its location.

Implementation ideas

No response

[quarkus-bot[bot]]

/cc @Sgitario, @geoand, @iocanel, @manovotn, @mkouba

[Sgitario]

So, you suggest two enhancements:

1. Allow to map properties by the secret name and namespace

So, in the case we have the same property keys, we can select from what secret we should take the property from. As you state, now all the properties are mixed up. For example, having these two secrets:

• secret: s1, namespace: ns1

  app: val1 

• secret: s2, namespace: ns2

  app: val99

Using both secrets, the value for the property app will be overwritten with the latter secret.

I purpose to continue doing so, so we avoid introducing breaking changes, but adding the prefixed properties:

s1/app=v1
ns1/s1/app=v1
s2/app=v99
ns2/s2/app=v99

So, you can inject the properties using the secret name and/or the namespace too. wdyt? @lordofthejars @geoand @iocanel

2. Allow to mount secrets as volumes.

I think this is already supported. See here: https://quarkus.io/guides/deploying-to-kubernetes#secret-volumes.

[lordofthejars]

Yes, the idea is to have a format to refer to keys in case of conflict. About point 2, yes, it's supported, but it's, in my opinion, a workaround for what I am mentioning; as if I am using Kubernetes Configuration extension to read secrets, shouldn't all secrets read in the same way? Just to not have two different ways in the same app.

[Sgitario]

Yes, the idea is to have a format to refer to keys in case of conflict. About point 2, yes, it's supported, but it's, in my opinion, a workaround for what I am mentioning; as if I am using Kubernetes Configuration extension to read secrets, shouldn't all secrets read in the same way? Just to not have two different ways in the same app.

About 2, ok for making it easier to use +1

Let's split the issue into two parts. The point 1. is addressed by https://github.com/quarkusio/quarkus/issues/27434.