common-workflow-language / cwltool

Common Workflow Language reference implementation
https://cwltool.readthedocs.io/
Apache License 2.0
332 stars 230 forks source link

InitialWorkDir listing with expression returning a directory: TypeError #227

Open mr-c opened 7 years ago

mr-c commented 7 years ago

initial-empty-directory.cwl

class: CommandLineTool
cwlVersion: v1.0
requirements:
  - class: InlineJavascriptRequirement
  - class: ShellCommandRequirement
  - class: InitialWorkDirRequirement
    listing: |
      ${return {'type': 'array', 'items': [ inputs.empty ]};}

inputs:
  empty: Directory

outputs:
  outlist: stdout

arguments: ["find", ".",
  {shellQuote: false, valueFrom: "|"},
  "sort"]
/home/michael/cwltool/env/bin/cwltool 1.0.20161110155008
Resolved 'initial-empty-directory.cwl' to 'file:///home/michael/common-workflow-language/v1.0/v1.0/initial-empty-directory.cwl'
Parsed job order from command line: {
    "id": "initial-empty-directory.cwl", 
    "empty": {
        "class": "Directory", 
        "location": "file:///home/michael/common-workflow-language/v1.0/v1.0/empty"
    }, 
    "job_order": null
}
[job initial-empty-directory.cwl] initializing from file:///home/michael/common-workflow-language/v1.0/v1.0/initial-empty-directory.cwl
[job initial-empty-directory.cwl] {
    "empty": {
        "basename": "empty", 
        "listing": [], 
        "class": "Directory", 
        "location": "file:///home/michael/common-workflow-language/v1.0/v1.0/empty"
    }, 
    "job_order": null
}
[job initial-empty-directory.cwl] path mappings is {
    "file:///home/michael/common-workflow-language/v1.0/v1.0/empty": [
        "file:///home/michael/common-workflow-language/v1.0/v1.0/empty", 
        "/tmp/tmp61L6Oy/stga03dfc90-d052-48d8-8438-888bdbf30616/empty", 
        "Directory"
    ]
}
[job initial-empty-directory.cwl] command line bindings is [
    {
        "position": [
            0, 
            0
        ], 
        "datum": "find"
    }, 
    {
        "position": [
            0, 
            1
        ], 
        "datum": "."
    }, 
    {
        "shellQuote": false, 
        "position": [
            0, 
            2
        ], 
        "valueFrom": "|"
    }, 
    {
        "position": [
            0, 
            3
        ], 
        "datum": "sort"
    }
]
[job initial-empty-directory.cwl] /tmp/tmpQmQtpz$ /bin/sh \
    -c \
    'find' '.' | 'sort' > /tmp/tmpQmQtpz/62560fea-6a41-4d73-9b2d-9665b454f832
Exception while running job
Traceback (most recent call last):
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/job.py", line 232, in run
    self.outdir, separateDirs=False)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 141, in __init__
    self.setup(dedup(referenced_files), basedir)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 181, in setup
    self.visit(fob, stagedir, basedir)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 157, in visit
    self.visitlisting(obj.get("listing", []), tgt, basedir)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 146, in visitlisting
    tgt = os.path.join(stagedir, ld["basename"])
TypeError: string indices must be integers
[job initial-empty-directory.cwl] completed permanentFail
[job initial-empty-directory.cwl] {}
Final process status is permanentFail
[job initial-empty-directory.cwl] Removing input staging directory /tmp/tmp61L6Oy
[job initial-empty-directory.cwl] Removing temporary directory /tmp/tmpCX1r9v
[job initial-empty-directory.cwl] Removing empty output directory /tmp/tmpQmQtpz
Workflow error, try again with --debug for more information:
  Process status is ['permanentFail']
Traceback (most recent call last):
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/main.py", line 710, in main
    **vars(args))
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/main.py", line 238, in single_job_executor
    raise WorkflowException(u"Process status is %s" % (final_status))
WorkflowException: Process status is ['permanentFail']
r06942072 commented 5 years ago

Dear @mr-c mr-c Could I ask what is the pipe | in listing section meaing? Thank you :)

mr-c commented 5 years ago

Hello @r06942072 , that is a YAML Block scalar indicator to do line wrapping. See http://yaml.org/refcard.html , https://learnxinyminutes.com/docs/yaml/ and http://yaml.org/spec/1.2/spec.html#Block