search

multiply-org / multiply-core

The core functionality of the MULTIPLY platform
2 stars 5 forks source link

How to specify configuration? #3

Open bulli92 opened 7 years ago

bulli92 commented 7 years ago

By implementing the SAR preprocessor, I recognized that it is quite important that we soon agree on how we store and pass the general configuration information between the core and the individual sub-modules.

In general I would be in favor that each sub-module only gets a single argument. This makes it much easier if things evolve over time. IN principle I see two options a) configuration as dictionary b) configuration in its own class

Thus for a) the call to a sub-module would look like

config = {'region' : {'some list off coordinates'}, 't_start' : date time.datetime(2000,1,5), t_stop : date time.datetime(2005,11,2)}

S = SARPreProcessor(config)
S. do_something()

For option b) it would be more like

C = Config(t_start='2006,1,1', t_stop='2011,7,31'). # configuration stored as attributes
S = SARPreProcessor(C)
S. do_something()

Both options are extendable. A dictionary can be easily stored on some file (e.g. YML). A class on the other hand can have methods that might help at some stage. Storing the config is also possible through a save function.

Thus, what do people think what option we should choose?

TonioF commented 7 years ago

Here, the discernation between command line interfaces and python API interfaces is very important. When a component is called via command line, we cannot pass a python object to it. We therefore need to have this config as a file (in YAML). When the file has been passed, the component-internal use is not of interest to the other modules anymore. I do not favor any of the above suggestions (dict or dedicated class).

TonioF commented 7 years ago

I have created a wiki page about the config file: https://github.com/multiply-org/multiply-core/wiki/Config-file . Creating this, I stumbled upon the issue whether all parameters should / can be included in the config file. E.g., the EO product that shall serve as input might not be known at the time when the config file is created or we might want to iterate over one parameter. To this, I propose that every component shall expect config files setting all parameters, but also offer command line input parameters. When a parameter is declared both in the config file and as an additional CLI parameter, the CLI value will be used.