common-workflow-language / cwltool

Common Workflow Language reference implementation
https://cwltool.readthedocs.io/
Apache License 2.0
332 stars 229 forks source link

InitialWorkDirRequirement fails to create an empty listing.entryname file #1076

Open azat-badretdin opened 5 years ago

azat-badretdin commented 5 years ago

Expected Behavior

empty listing.entryname file is expected to be created when inputs.annotation is NULL

Actual Behavior

empty listing.entryname file is not created

Workflow Code

cwlVersion: v1.0 
label: 'test_JS'
# internal NCBI location for our reference:  https://bitbucket.ncbi.nlm.nih.gov/users/badrazat/repos/jira/commits/a391294d18cf9d88168f61608eea591a230669a3#vc-PGAPX-313-Unable-to-open-the-manifest-for-/test_cwltool.cwl

class: CommandLineTool
requirements:
    - class: InlineJavascriptRequirement
    - class: InitialWorkDirRequirement
      listing:
        - entryname: annotation.mft
          entry: ${ if ( inputs.annotation == null ) { return ''; } else { return inputs.annotation.path; } }

baseCommand: cat
inputs:
  annotation:
    type: File?
  annotation_manifest:
    type: string
    default: annotation.mft
    inputBinding:
        position: 1 
outputs: []

Full Traceback

$ cwltool --debug --tmpdir-prefix ./tmpdir/  --leave-tmpdir  --tmp-outdir-prefix ./tmp-outdir/ --copy-outputs  --outdir ./outdir  test_cwltool.cwl 
/home/badrazat/tmp2/cwltool-git/cwltool/venv/bin/cwltool 1.0.20190228155703
Resolved 'test_cwltool.cwl' to 'file:///home/badrazat/bitbucket/repos/jira/vc-PGAPX-313-Unable-to-open-the-manifest-for-/test_cwltool.cwl'
Parsed job order from command line: {
    "id": "test_cwltool.cwl",
    "annotation": null,
    "annotation_manifest": "annotation.mft"
}
[job test_cwltool.cwl] initializing from file:///home/badrazat/bitbucket/repos/jira/vc-PGAPX-313-Unable-to-open-the-manifest-for-/test_cwltool.cwl
[job test_cwltool.cwl] {
    "annotation": null,
    "annotation_manifest": "annotation.mft"
}
[job test_cwltool.cwl] path mappings is {}
[job test_cwltool.cwl] command line bindings is [
    {
        "position": [
            -1000000,
            0
        ],
        "datum": "cat"
    },
    {
        "position": [
            1,
            "annotation_manifest"
        ],
        "datum": "annotation.mft"
    }
]
[job test_cwltool.cwl] initial work dir {}
[job test_cwltool.cwl] /home/badrazat/bitbucket/repos/jira/vc-PGAPX-313-Unable-to-open-the-manifest-for-/tmp-outdir/bg7ji6s1$ cat \
    annotation.mft
cat: annotation.mft: No such file or directory
Could not collect memory usage, job ended before monitoring began.
[job test_cwltool.cwl] completed permanentFail
[job test_cwltool.cwl] {}
[job test_cwltool.cwl] Removing input staging directory /tmp/tmp4w43_qvl
{}
Final process status is permanentFail

Your Environment

whlavina commented 5 years ago

I believe the bug is in checking whether entry is present: in Python, an empty string is False, so the following check explicitly prevents files with empty content. I think empty files should be allowed, and if you need a check, it should be for presence of a value, i.e. not None:

--- command_line_tool.py
+++ FIXED_command_line_tool.py
@@ -436,7 +436,7 @@
                             else:
                                 et["entryname"] = None
                             et["writable"] = t.get("writable", False)
-                            if et[u"entry"]:
+                            if et[u"entry"] is not None:
                                 ls.append(et)
                     else:
                         initwd_item = builder.do_eval(t)
whlavina commented 5 years ago

Note that the check for evalutates-to-false entry was added as part of issue #879, apparently to handle null values. An empty string is not a null value, so I think my recommended fix doesn't conflict with the intent of the additional check.