RTS test system to SpineOpt

[jkiviluo]

In order to test SpineOpt against the RTS test system and also to increase SpineOpt's visibility, we should make it possible to run RTS test system in SpineOpt and include SpineOpt in https://github.com/gridmod/rts-gmlc

[LouisFliche]

• An initial partial translation of the RTS GMLC database has been carried out here. A good part of the database has been translated, but there are problems in the translated database (some due to version problems, others due to errors). SpineOpt does not run on this database. It is difficult to try and complete or repair this database, as it has been translated by hand, so the calculations made and the order and manner in which the database was translated are not easily retrieved.

• I tried to translate this RTS database, this time using a script, so that the process would be automated and reproducible. To do this, I've tried to develop a basic database modification tool in Spine-Toolbox. In a way, it's a rough outline of the data transformer we'd like to develop (see this issue). The operation of this workflow is more precisely described in the other issue.

• I was able to translate part of the database thanks to this. Everything that consisted of systematic operations on parameters (rename, multiply by a constant factor), and between parameters: creation of a parameter equal to the sum of two others, multiplication, division, maximum between two parameters etc. could be carried out. The Spine-Toolbox project RTS_To_SpineOpt_Basis presents this part of the workflow.

• However, translating RTS into a SpineOpt format database also required operations on the different values of the same parameter: finding maximum of the values taken by this parameter, selection of parameters corresponding only to certain objects (i.e. selection of a parameter as a function of the value of another parameter), and so on. For example, we want to be able to select only one type of units, to then create some parameters or make specific operations. To do this, I added a "Filters and operations on columns" section to the previous workflow, upstream of the operations between/on the parameters described above. The Spine-Toolbox project RTS_To_SpineOpt_with_filters presents this part of the workflow.

• However, this is only a very limited example. In fact, we would like to be able to perform all the basic operations on databases: joins, selections, etc. (see the issue mentioned above). The question then arose of how to proceed and how to take account of current changes in database management. It seemed better to wait until we had a dedicated tool before finalising the translation of RTS into a SpineOpt data format.