common-workflow-language / cwltool

Common Workflow Language reference implementation
https://cwltool.readthedocs.io/
Apache License 2.0
335 stars 231 forks source link

self.path fails for File w/ location #452

Closed mr-c closed 7 years ago

mr-c commented 7 years ago

Expected Behavior

Resolved 'break.cwl' to 'file:///home/michael/cwltool/break.cwl'
[job break.cwl] /tmp/tmpW1GmoS$ echo \
    /tmp/tmpyjzHPE/stgf8e0802a-9c8b-4c83-8708-a9deb3784d27/index.html
/tmp/tmpyjzHPE/stgf8e0802a-9c8b-4c83-8708-a9deb3784d27/index.html
[job break.cwl] completed success
{}
Final process status is success

Actual Behavior

/home/michael/cwltool/env/bin/cwltool 1.0.20170629171139
Resolved 'break.cwl' to 'file:///home/michael/cwltool/break.cwl'
Got workflow error
Traceback (most recent call last):
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/main.py", line 250, in single_job_executor
    for r in jobiter:
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/draft2tool.py", line 304, in job
    builder.pathmapper = self.makePathMapper(reffiles, builder.stagedir, **make_path_mapper_kwargs)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/draft2tool.py", line 192, in makePathMapper
    return PathMapper(reffiles, kwargs["basedir"], stagedir)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 179, in __init__
    self.setup(dedup(referenced_files), basedir)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 227, in setup
    self.visit(fob, stagedir, basedir, copy=fob.get("writable"), staged=True)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 216, in visit
    self.visitlisting(obj.get("secondaryFiles", []), stagedir, basedir, copy=copy, staged=staged)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/schema_salad/sourceline.py", line 152, in __exit__
    raise self.makeError(six.text_type(exc_value))
ValidationException: in.yaml:3:3: [Errno 2] No such file or directory: '/home/michael/cwltool/http://www.commonwl.org/index.html'
Workflow error, try again with --debug for more information:
in.yaml:3:3: [Errno 2] No such file or directory:
             '/home/michael/cwltool/http://www.commonwl.org/index.html'

Workflow Code

break.cwl

cwlVersion: v1.0
class: CommandLineTool
inputs:
  test:
    type: File
    inputBinding:
      valueFrom: $(self.path)

baseCommand: echo

outputs: []

in.yaml

test:
  class: File
  location: http://www.commonwl.org/index.html

Full Traceback

/home/michael/cwltool/env/bin/cwltool 1.0.20170629171139
Resolved 'break.cwl' to 'file:///home/michael/cwltool/break.cwl'
[job break.cwl] initializing from file:///home/michael/cwltool/break.cwl
[job break.cwl] {
    "test": {
        "class": "File", 
        "location": "http://www.commonwl.org/index.html", 
        "basename": "index.html", 
        "nameroot": "index", 
        "nameext": ".html"
    }
}
Got workflow error
Traceback (most recent call last):
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/main.py", line 250, in single_job_executor
    for r in jobiter:
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/draft2tool.py", line 304, in job
    builder.pathmapper = self.makePathMapper(reffiles, builder.stagedir, **make_path_mapper_kwargs)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/draft2tool.py", line 192, in makePathMapper
    return PathMapper(reffiles, kwargs["basedir"], stagedir)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 179, in __init__
    self.setup(dedup(referenced_files), basedir)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 227, in setup
    self.visit(fob, stagedir, basedir, copy=fob.get("writable"), staged=True)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/pathmapper.py", line 216, in visit
    self.visitlisting(obj.get("secondaryFiles", []), stagedir, basedir, copy=copy, staged=staged)
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/schema_salad/sourceline.py", line 152, in __exit__
    raise self.makeError(six.text_type(exc_value))
ValidationException: in.yaml:3:3: [Errno 2] No such file or directory: '/home/michael/cwltool/http://www.commonwl.org/index.html'
Workflow error, try again with --debug for more information:
in.yaml:3:3: [Errno 2] No such file or directory:
             '/home/michael/cwltool/http://www.commonwl.org/index.html'
Traceback (most recent call last):
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/main.py", line 791, in main
    **vars(args))
  File "/home/michael/cwltool/env/local/lib/python2.7/site-packages/cwltool/main.py", line 262, in single_job_executor
    raise WorkflowException(Text(e))
WorkflowException: in.yaml:3:3: [Errno 2] No such file or directory: '/home/michael/cwltool/http://www.commonwl.org/index.html'

Your Environment

mr-c commented 7 years ago

@kapilkd13 What's your progress?

kapilkd13 commented 7 years ago

Hi @mr-c This error was due to http link was given as input. https://github.com/common-workflow-language/cwltool/pull/507 This PR addresses this issue also. I tested it on my local system and this workflow works after the http input PR. 😄

mr-c commented 7 years ago

Great! I forward to #507 being ready for merging

kapilkd13 commented 7 years ago

A unit test is remaining though. I will try to add it tonight.

mr-c commented 7 years ago

Fixed by #507