Amend the total number of jobs in a workflow so that when Snakemake
detects that one or more jobs can be restored from the workspace (same
workflow caching) they are not considered in the total count, reflecting
Snakemake's local behaviour.
Closes #62.
How to test
roofit example
Run the workflow normally with reana-client run -w roofit-snakemake -f ./reana-snakemake.yaml. The total number of jobs should be 2/2 (as usual)
Restart the workflow in the same workspace with reana-client restart -w roofit-snakemake. The total number of jobs should be 0/0, because the workflow is entirely restored from the workspace.
Change something in code/fitdata.C (such as a comment) and upload it to the workspace with reana-client upload -w roofit-snakemake code/fitdata.C
Restart the workflow in the same workspace with reana-client restart -w roofit-snakemake. The total number of jobs should be 1/1, because only the fitdata step is executed (gendata is restored from the "local cache").
cms-h4l example
Run the workflow normally with reana-client run -w cms-h4l. The total number of jobs should be 4/4 (as usual)
Restart the workflow in the same workspace with reana-client restart -w cms-h4l. The total number of jobs should be 0/0, because the workflow is entirely restored from the workspace.
Remove the Higgs4L1file.root (which is the input for the make_plot step, generated by analyze_mc) with reana-client rm -w cms-h4l-snakemake.1.4 results/Higgs4L1file.root and the final output plot: reana-client rm -w cms-h4l-snakemake.1.4 results/mass4l_combine_userlvl3.pdf
Restart the workflow in the same workspace with reana-client restart -w cms-h4l. The total number of jobs should be 2/2, because the analyze_mc step should regenerate the Higgs4L1file.root file and the make_plot step should create the final output plot.
Amend the total number of jobs in a workflow so that when Snakemake detects that one or more jobs can be restored from the workspace (same workflow caching) they are not considered in the total count, reflecting Snakemake's local behaviour.
Closes #62.
How to test
roofit
examplereana-client run -w roofit-snakemake -f ./reana-snakemake.yaml
. The total number of jobs should be 2/2 (as usual)reana-client restart -w roofit-snakemake
. The total number of jobs should be 0/0, because the workflow is entirely restored from the workspace.code/fitdata.C
(such as a comment) and upload it to the workspace withreana-client upload -w roofit-snakemake code/fitdata.C
reana-client restart -w roofit-snakemake
. The total number of jobs should be 1/1, because only thefitdata
step is executed (gendata
is restored from the "local cache").cms-h4l
examplereana-client run -w cms-h4l
. The total number of jobs should be 4/4 (as usual)reana-client restart -w cms-h4l
. The total number of jobs should be 0/0, because the workflow is entirely restored from the workspace.Higgs4L1file.root
(which is the input for themake_plot
step, generated byanalyze_mc
) withreana-client rm -w cms-h4l-snakemake.1.4 results/Higgs4L1file.root
and the final output plot:reana-client rm -w cms-h4l-snakemake.1.4 results/mass4l_combine_userlvl3.pdf
reana-client restart -w cms-h4l
. The total number of jobs should be 2/2, because theanalyze_mc
step should regenerate theHiggs4L1file.root
file and themake_plot
step should create the final output plot.