Accumulated fixes and minor changes from BSC

[jmfernandez]

Since past December 2021 @lcodo , @txellferret and I have been accumulating fixes and minor changes related to the QfO benchmarking workflow, mainly related to the reproducibility. These changes can be described as:

• Changing the tag used in the workflows from latest to the specific number.
• When a workflow parameter represents either a file or a directory, label it as such through all the steps where it is involved, so Nextflow stages them properly in docker-in-docker workflow executions.
• Better scheduling of which workflow steps are executed.
• Other minor fixes.

[alpae]

Hi @jmfernandez ,

thanks for the PR. I've merged it, but I realized that several changes were reverted by you. I don't think that the current version is what you run at OpenEBench - the VGNC benchmark for example wouldn't run like this (it is missing the sqlite database).

Do you have some additional changes lying around? Otherwise, I will have to go through the code and add back some of the missing features.

Cheers Adrian

[jmfernandez]

Hi @alpae , I / we have no additional changes right now. Sorry for the unintended changes related to the VGNC benchmark . I have just checked both QfO_2020 VRE tools metadata:

• For QfO_2020_extended we are currently using https://github.com/jmfernandez/benchmark-webservice/tree/02d8da8815ba2e0f32adb43eedf292ff0cf8894a
• Meanwhile for "old" QfO_2020 we are currently using https://github.com/inab/qfo-benchmark-webservice/tree/29f3d3e58b0ff0d14734aac50c9e15b3c86f7e33

Although a bit off topic, the example for the efforts to have a reproducible command line test uses https://github.com/jmfernandez/benchmark-webservice/tree/4da3128a1ab3901b6eddcaf14599b23b9700f909 , and it also has to generate a "reasonable" input file which is not discarded due the datasets (see scripts at https://github.com/inab/vre-process_nextflow-executor/tree/master/tests/QfO)