Separate loading of application configurations and secrets into more than one file

[rdsarvar]

The goal of this request is to have the loading of basic configurations and secrets split up to provide a separation of concerns.

An example usage where this would be required would be running Trino Gateway in Kubernetes using the provided Helm chart. Currently, all of the configurations/secrets are merged and mounted to the container as a single YAML file. In order to follow this strategy that means:

• In a secured K8s cluster, you are unable to introspect the configuration being mounted to the pod unless you have elevated permissions to view Secret objects. Usually, ConfigMap resources are R/O to majority of users.
• Your deployment strategy requires elevated permissions to view Secret objects so that a lookup can be done on pre-existing secrets which will be merged into the standalone secret
  • A widely used deployment tool ArgoCD does not support the lookup strategy.

Opening this issue to spur conversations on better strategies for configuring the application.

An initial suggestion could be leveraging Jackson's ObjectMapper with YAMLFactory and the readerForUpdating functionality. This could support loading of multiple configurations by the following strategy:

1. Update the application to allow > 1 configuration file paths through arguments.
2. Leverage Jackson's ObjectMapper with YAMLFactory and initialize/update the application config using the mapper's readerForUpdating method.

I'm not 100% sure how Trino is handling this in their Kubernetes deployment but it would be interesting to see if we can follow suit.

[mosabua]

We currently have it all in one to keep it simple. Once the airlift refactor is in we plan to work with Trino and airlift projects to figure out good approach. Our need is kinda different since we use yaml for nested configs quite a bit and Trino/airlift currently dont (yet). So maybe we do something like

• config.properties like in trino for global config
• log.properties for logging package config
• jvm.config for JVM start configuration
• maybe node.properties but ideally not since all TGW nodes should be the same and stateless
• some way to get more yaml config in
• some better way for secrets .. similar to how in trino we can use env.* and maybe more stuff

We can discuss more in the next sync for starters. Also fyi and for sanity check @willmostly @Chaho12 @vishalya and @oneonestar

[oneonestar]

I think env variable is a good way. We can use env: secretKeyRef to set env. variable from secret in K8S YAML. After migrate to Airlift, settings in httpConfig section are allowed to use env variable (impl. by Airlift). We might want to add the same function to other sections.

httpConfig:
  http-server.https.keystore.path: cert.jks
  http-server.https.keystore.key: ${ENV:KEYSTORE_KEY}

Also, the plain text password in presetUsers section should be gone. We could use something similar to password file in Trino.

If we decide to pull in io.trino.spi.*, we could reuse FileAuthenticator from Trino.