Mount Secrets as files instead of env vars in the Operator

[vmuzikar]

Quarkus dist options should be read from file(s).

[vmuzikar]

Related: #15830

[shawkins]

KeycloakDistConfigurator seems to allow for multiple secrets, but --config-file seems to specify only a single file. Are there mechanisms for picking up the config from multiple files?

[vmuzikar]

@shawkins It's certainly possible to use multiple config files but we do not expose this option.

However, I believe leveraging SmallRye Secret Keys Expressions with some (custom) plain text files handler might be more suitable approach here.

[shawkins]

However, I believe leveraging SmallRye Secret Keys Expressions with some (custom) plain text files handler might be more suitable approach here.

I missed that the existing --config-file can accept a directory, but it's not recursive so that's not applicable to multiple mounts.

Next up would be quarkus.config.locations - which could be set at runtime to include multiple locations pointing to the mounted secrets. However it's problematic to support this with the existing SecretKeySelectors that specify a key - it would then take users putting the secrets in a file format instead:

stringData:
  username.properties: |
    username: ...

Then we'd still need some mapping of username to db-username - which if we did via an env property we'd be no better off than when we started. So we'd have to inject some other bit of configuration to handle that mapping.

Alternatively there is https://smallrye.io/smallrye-config/2.9.0/config-sources/filesystem/ which would require injecting additional config that looked like:

smallrye.config.source.file.locations=file:/mnt/database/username,file:/mnt/database/password
db-username=${key1}
db-password=${key2}

But of course that means you must use unique key names - they cannot conflict with other or any other existing property. That dependency is not currently in keycloak.

Or as you point out you could add a custom secret key hanlder so that the secrets could be mounted in the existing flat form, and then inject additional configuration that looked like:

db-username=${custom-file::/mnt/database/username/key1}
db-password=${custom-file::/mnt/database/password/key2}

Which requires adding that handler implementation to the keycloak image, but the quarkus docs https://quarkus.io/guides/config-reference#secret-keys-expressions do call out that they only support one of the existing mechanisms - we'd have to ask if there were any objection to supporting a new custom one.

Or the keycloak database properties could accept files (like the tls configuration does) - db-username-file, db-password-file - that would allow things to work with just the mounts and not introduce additional config handling.