reanahub / reana-job-controller

REANA Job Controller
http://reana-job-controller.readthedocs.io/
MIT License
2 stars 38 forks source link

htcondorcern: snakemake multiline shell scripts are not parsed as expected #470

Open mdonadoni opened 1 month ago

mdonadoni commented 1 month ago

Consider the following changes to reana-demo-helloworld

diff --git a/workflow/snakemake/Snakefile b/workflow/snakemake/Snakefile
index 4532d3c..7538c51 100644
--- a/workflow/snakemake/Snakefile
+++ b/workflow/snakemake/Snakefile
@@ -28,7 +28,7 @@ rule helloworld:
     container:
         "docker://docker.io/library/python:2.7-slim"
     shell:
-        "python {input.helloworld} "
-        "--inputfile {input.inputfile} "
-        "--outputfile {output} "
-        "--sleeptime {params.sleeptime}"
+        """
+echo 'first'
+echo 'second'
+"""
diff --git a/workflow/snakemake/Snakefile-htcondorcern b/workflow/snakemake/Snakefile-htcondorcern
index 9367a76..dfcfd78 100644
--- a/workflow/snakemake/Snakefile-htcondorcern
+++ b/workflow/snakemake/Snakefile-htcondorcern
@@ -16,7 +16,7 @@ rule helloworld:
     container:
         "docker://docker.io/library/python:2.7-slim"
     shell:
-        "python {input.helloworld} "
-        "--inputfile {input.inputfile} "
-        "--outputfile {output} "
-        "--sleeptime {params.sleeptime}"
+        """
+echo 'first'
+echo 'second'
+"""

These are the logs when running reana-snakemake.yaml (snakemake on k8s):

job: :
 first
second

Completed

In this case, only first and second are printed, as expected.

These are the logs when running reana-snakemake-htcondorcern.yaml (snakemake on htcondorcern):

command to execute:
echo 'first' echo 'second'first echo second

In this second case, first echo second is printed, as the second echo command is passed as a parameter to the first one.

We should check whether multiline """ strings are supported by Snakemake (but they should, as the k8s example works) and why HTCondor behaves differently than k8s.