Should GALAXY_CONFIG_* env variables work for all config options?

[innovate-invent]

This is a continuation of the discussion here https://github.com/galaxyproject/galaxy/pull/11760

Currently not all configuration options are able to be managed by GALAXY_CONFIG_* env variables. Particularly logging, some database_engine_option_*, or any option that requires more than a primitive type.

Should all config options be supported? Should support for complex types be uniform with the other config options?

It was proposed that a alternate mechanism be provided for complex values, but I see more value in a uniform interface.

Also, can someone tell me the logic behind GALAXY_CONFIG_OVERRIDE_*? What real world situation would you want the runtime environment to not override the static configuration?

[jmchilton]

What real world situation would you want the runtime environment to not override the static configuration?

I felt for applications like Planemo and Docker images it made some sense to provide defaults but allow the user's file configurations to take precedence. For applications like driving tests and such, I frequently do want to override the contents of the file though. This has seemed to work out well, perhaps it was overkill though - I'm not really in a position to provide a detailed pros and cons list. If as an admin you think they should just all be GALAXY_CONFIG_OVERRIDE_* variants - then just use those.

[jmchilton]

Should support for complex types be uniform with the other config options?

This question is asked in a very leading way. Those options are plain strings when read from an ini file and complex types when read from YAML. They are already not uniform. I prefer uniformity in how they parsed for the reasons laid out by @ic4f in that issue, the logic seems less error prone and easier to convey to admins and developers.

Should all config options be supported?

If there are gaps in what can be controlled that way - by all means I'd love to see them addressed. Some things are parsed before the config object is created, some things are read before Galaxy is even loaded (e.g. by the app server or middleware) so this is difficult to solve in general. But if there are particular options you want parsed more aggressively - that seems fine and good. I know I've encountered frustration when the application server was reading something I wanted to have control over, but the fancy environment parsing only handles the "application configuration".

[innovate-invent]

I felt for applications like Planemo and Docker images it made some sense to provide defaults but allow the user's file configurations to take precedence. For applications like driving tests and such, I frequently do want to override the contents of the file though. This has seemed to work out well, perhaps it was overkill though - I'm not really in a position to provide a detailed pros and cons list.

From what I have seen, the images are supposed to be configured via the env variables.

If as an admin you think they should just all be GALAXY_CONFIG_OVERRIDE_* variants - then just use those.

The underlying issue with the direction #11760 took is that it is packing data into the variable name. I don't think it is correct and I am struggling to find a solid reason to keep adding complexity to it rather than just have the GALAXY_CONFIG_* prefix and nothing more.

Those options are plain strings when read from an ini file

I had actually forgotten about ini files. Aren't they depreciated for this reason?

Some things are parsed before the config object is created, some things are read before Galaxy is even loaded (e.g. by the app server or middleware) so this is difficult to solve in general.

Are you referring to the uwsgi config? I think that will disappear with FastAPI