Set up prior engine framework

[TonioF]

Prior code shall be presented in the form of plug-ins: In this way, we avoid having to change the prior engine code everytime we introduce a new prior. It would facilitate the adding of new priors.

[tramsauer]

User defined priors

My idea was to stick with the configuration file approach and have the variables which are to be inferred and their respective prior data information (like directory) in the config files and let the user change the settings therein via read/display/change/write-config methods.

Status

• I've set up a first draft for methods to adjust the prior engine's configuration file to include user defined priors in my fork: https://github.com/RaT0M/prior-engine/blob/dev/tom/multiply_prior_engine/user_prior.py

  • It so far allows the user to basically add/define a directory, that holds the user's prior data which is to be included in the inference process [and is in a compatible format..). compatible data so far still only is: 2-layer (mean/unc) global gdal-(vrt)-file.

  • There is a simple CLI included which could/should/will be added as command line entry point in the prior engine setup.py.

TODO:

• Add Flexibility: One of the missing points would be to include some sort of conversion tools to add more flexibility here (e.g. to read in tabular data). this could be very extensive..
• So far, only scaffolds of methods to unselect/delete priors are in place, but will shortly be completed.

--> What do you think of this approach? I am not experienced with 'framework design'..so this could go in a totally wrong direction. Furthermore, this is not really a plug-in system..

If this is an acceptable way to go, I'll merge the branch to multiply-org..

[tramsauer]

updated code. delete/show is implemented but not yet in CLI.

[TonioF]

Thanks for working on this!

So, to recapitulate: You have written an extension that allows users to add a .vrt-file. The prior engine recognizes which variables are in the data and enables users to use them in the MULTIPLY workflow. Is this correct?

If so, I'd say this is one generic prior which we need - others would be generic priors to read data from, say, excel tables or ascii files. The framework, however, would go beyond that. It would allow users to not only add their own data, but to write their own code which can be interpreted by the prior engine. I don't think your approach covers that, or did you have plans to integrate this?

We should also clarify where the data is coming from. If it is EO data, it should be saved in the data store and passed on from the Data Access Component. If it is small auxiliary data (e.g., some text file) it can become part of the prior engine.

[tramsauer]

All questions are further discussed in other issues:

So, to recapitulate: You have written an extension that allows users to add a .vrt-file. The prior engine recognizes which variables are in the data and enables users to use them in the MULTIPLY workflow. Is this correct?

==> See #21 for CLI & #24 for user prior input methods

If so, I'd say this is one generic prior which we need - others would be generic priors to read data from, say, excel tables or ascii files. The framework, however, would go beyond that. It would allow users to not only add their own data, but to write their own code which can be interpreted by the prior engine. I don't think your approach covers that, or did you have plans to integrate this?

==> See #24 for different sorts of user prior input. ==> See #26 for framework introduction

We should also clarify where the data is coming from. If it is EO data, it should be saved in the data store and passed on from the Data Access Component. If it is small auxiliary data (e.g., some text file) it can become part of the prior engine.

==> See #27