search

TrueBlocks / trueblocks-core

The main repository for the TrueBlocks system
https://trueblocks.io
GNU General Public License v3.0
1.06k stars 202 forks source link

Replace viper for reading configuration #3783

Open dszlachta opened 5 months ago

dszlachta commented 5 months ago

We currently use a package called Viper to read the configuration (TOML file and environment variables). However, it has one (at least) irritating issue - it only reads environment variables if there's a default value registered for the configuration key. It would be hard and pointless to have such defaults defined for "dynamic" keys like API providers access keys and possible future settings.

It also works differently when there is and when there isn't a TOML file, which makes debugging configuration issues more problematic.

I suggest we move to koanf - I've been using it for Key and didn't see any issues. Of course the first part of this task would be to double check that the environmental variable problem doesn't occur with koanf.

tjayrush commented 5 months ago

I am 100% in support of making the configuration file management better. I've been struggling with it for a very long time.

One thing that is very important to me is that the data structure we use to store all the configuration data is auto-generated. There's a large number of reasons that I can go in to: documentation, auto-generation of testing code, auto-generation of front end code for setting config data in App, consistency of code, binary caching (i.e., backing store), etc.

I'm talking about just the data structure, here. Not which library we use.

dszlachta commented 5 months ago

Do you mean you'd like to have Config type defined in goMaker/templates/classDefinitions/fields/config.csv?

tjayrush commented 5 months ago

Do you mean you'd like to have Config type defined in goMaker/templates/classDefinitions/fields/config.csv?

Yes. That's exactly what I mean. (You also need goMaker/templates/classDefinitions/config.toml

(Check out an example such as lightBlock.toml and fields/lightBlocks.csv.)