Custom URI based loading of sources extension idea

[agentgt]

So I still have not fully replaced our home grown config with avaje-config because of one feature that I can't decide whether to just work around or have it implemented in avaje-config.

What we have in our system that Avaje does not is we have a

• parser config extension just like Avaje generally based on the file name resource path extension.
• a filter properties extension - not needed really
• loading flags which I talked about - not needed really
• URI schema based custom loading - used a lot!

Basically allow custom loading of resources using URIs. This may already be built with the resource loading SPI but I don't think so.

What I mean by that is we do things like using load.properties as an example:

load.properties=classpath:/somepath.properties env:///MY_PROJECT_PREFIX_DATABASE_/DATABASE_/ gcloud.meta:///

I think the classpath one is obvious and builtin but the env one is builtin for us but I would implement as an extension.

What the environment URI one does is find all environment variables starting with MY_PROJECT_PREFIX_DATABASE_ and replace that starting with DATABASE_. (it does occur to me that perhaps my real use case need is something that just transforms key names but let us just work through this more abstract feature).

Basically internally we have a Map of URI schema to Config loader which is SPI discovered.

// ignore names... you know the bikeshed avoidance drill

interface URILoadingConfigServiceProvider extends ConfigExtension {
  String getSchema();
  Config load(URI uri); // I don't know the exact avaje-config types but you get the idea.
}

Obviously no schema found and it does whatever the defaults are. I realize this adds some more bloat but I think it is not that bad given we have now consolidated service loading to a single call (thank you @SentryMan ).

[agentgt]

So my discussion with @SentryMan does not get lost in the abyss that is discord:

[13:52 - 13:58] agentgt:

to be honest it is the killer feature of our in house config system basically the loading of resources and the configuration of the loading itself is URI driven so you can use you know like URI query parameters etc And URI are designed for one line. Actually rainbowgum uses this as well just far less obvious The only tricky thing is deciding whether or not you want to pass in some sort of parser lookup Config load(URI uri, ParserLookupByExtensionOrMimeEtc thingy) otherwise the URI SPI just has to handle it all which maybe fine. The only bad thing is documenting URI format and parameters is less obvious than javadoc-ing some builder or getter/setter pojo

[13:58] SentryMan:

How does one get all the env variables with a certain prefix

[13:59 - 14:04] agentgt:

System.getenv gives you map. its a little messed up because snaphop-kvs basically passes an Environment thing to SPI calls so that System.getenv can be unit tested ah fuck I said my company's name oh well That is System.getenv() returns a map System properties also has similar Basically in our system it looks more like Config load(URI uri, ParserLookupByExtensionOrMimeEtc thingy, Environment env) but avaje-config probably does not need that complexity. Environment has shit like the classloader and facades for system properties / env, system clock etc. Its basically just to ease unit testing.

[14:08 - 14:18] agentgt:

When you do your load.properties split to list if its not a URI or a URI missing schema than you do the default loading mechanism otherwise you go check in your URI schema registry of that SPI Another awesome one we have is stdin... probably less useful for cloud stuff but useful for CLI cat some.properties | java -jar some.jar some.jar has load.properties=stdin:/// Its actually pretty useful for password sensitive stuff An analog is: https://docs.docker.com/reference/cli/docker/login/#password-stdin

cat ~/my_password.txt | docker login --username foo --password-stdin I think if stdin is not piped anything it won't block if you are worried about that

If you are wondering what format that stdin plugin parses that is why the parser thing is passed. load.properties=stdin:///?mimeType=text/json // I can't remember what we actually use but you get the idea. I can't remeber because we basically always use jul properties the only annoything btw is technically I think you are supposed to escape the : in jul properties but I haven't found it to be a problem. likewise for = java util properties is forgiving I guess

[14:22 - 14:26] agentgt:

I'm looking at the URI schemas we support and its a lot. So now I'm really confident if you implement this it would be a massive help for me to transition to avaje-config

CLASSPATH,
JAR,
HTTP,
HTTPS,
STDIN,
CMD,
ENV,
FILE,

that is just the builtin Oh guess what CMD does ... e.g. cmd:/// command like arguments For example I think to emulate -D like Maven where its passed after the .jar you can do

load.properties=cmd:/-D/

Sorry for poor formatting!