Closed Sinclert closed 4 years ago
Thank you @Sinclert , this is the PR we talked about over a private Zoom meeting. I tested it locally and works smoothly until the Pythia step. I think we can proceed with the merge. Thank you this is a crucial PR!
Problem
One of the issues that is interfering with the next goals that this project repository needs to achieve:
yadage-run
.is the writing of workflow output files in the same directory as static code and scripts. This causes problems because:
workdir
. The "workdir" is basically the folder where workflow steps can interchange files with each other. Now, due to this project's code and script into two independent folders (docker-madminer-ml and docker-madminer-physics), there were actually two "workdirs", which was invalid.Proposed solution
Thanks to the definition of an intermediate shell-script layer between the workflow specification and the Python code, the output folder is now parametrizable.
This PR introduces a new argument (
-o
) to all the scripts (both in docker-madminer-ml and docker-madminer-physics folders), which is used as the path to store files.Other additions:
755
(otherwise non-executable).__future__
syntax from Python3 files, which is Python2 legacy.yadage-run
by:initdir
option (for theinputs.yml
file).rootdir
option (for theworkflow.yml
andsteps.yml
files).The definition of
initdir
androotdir
was only possible after a private conversation with @lukasheinrich, as they were not explained in the Official Yadage documentation (will they soon be? 🤷♂️ )