Modular `config.properties` possible?

[KristofVDB1]

We have the shacl-validator running on a specific domain called applicatieprofielen. This domain contains the different properties inside the config.properties file. Our directory structure looks like this:

/resources/applicatieprofielen/config.properties

However, the config.properties file seems to be a static file containing all the properties for our API. From what I can grasp, this means that if we want to add a new config to this file, it requires a redeployment of the docker service with a new config. I tried looking into the docs to see if there is support for a "dynamic" config file, but did not immediately find it. My apologies if it does exist somewhere and I just missed it.

Is this something that you guys support out of the box or is this something that you guys have had to deal with in the past?

Thanks in advance!

Kind regards Kristof

[costas80]

Indeed the config.properties file is not meant to be updated and refreshed on the fly. It is read only once at startup. The closest thing to a "dynamic" configuration is if you don't point to SHACL shapes as local file references, but rather use remote URIs. In this case remotely loaded shapes are cached and refreshed automatically every hour. From what I understand however what you are going for is a way to add something like a new profile (a new validation type or option) and have this be automatically available. This is not possible I'm afraid but there are a couple solutions that could get you close:

Use a volume for the config

In case you are creating a custom image from the base isaitb/shacl-validator you could also use the base image directly and supply the configuration through a volume (see the image README):

docker run -d --name my-validator -p 8080:8080 \
             -v /validator/resources:/validator/resources/ \
             -e validator.resourceRoot=/validator/resources/  \
             isaitb/shacl-validator

Like this if you change your config.properties file you don't need to recreate your image but only do a container restart. This is not an "on-the-fly" update but it could be more interesting.

Use Kubernetes rolling updates

If you follow our validator production installation guide you'll see that we advise the use of Kubernetes to deploy your validator. Using Kubernetes you can update your validator instance(s) based on your new configuration and only make them available when ready. In fact this is exactly what we do for the validator instances we manage. The configuration per validator is maintained in separate GitHub repos from which we recreate the validator instances and bring them online when ready via rolling updates.

I know what I propose is not a dynamic reloading of the config.properties file but there is reason why we don't allow this. A configuration reload might take several seconds (especially if reading and caching remote shapes) during which time ongoing validations might run into inconsistencies (yes, things could be isolated when reloading but that would be quite complex). Also, if there are configuration errors we typically want to catch these at startup so that they don't go unnoticed.

[KristofVDB1]

Thanks for the quick feedback and trying to come up with a solution with us! I completely understand the reasoning behind not allowing a "dynamic" config-file.

A volume is for sure something worth checking out to see if that works for our setup. Some delay is more than perfect, doesn't have to be real-time. Main goal is mostly that it doesn't require a new image. As for Kubernetes, that will most likely be a bit of an overkill for our use case but it's good to keep in mind nonetheless.

I will close this issue!

Thanks again @costas80 !