"No such file or directory" in very simple example

[Keyeoh]

Hi everybody,

First of all I would like to congratulate you all for the amazing work you are doing with the CWL. It really seems a great idea to try to standardize practices which we are actually reimplementing almost continuously in our work (I am a bioinformatician).

I am beginning with the tools, just trying to replicate some of the simple examples (Hello world!) and finding out if I can do something similar with a command line R script, which is something I am currently very interested in. I already have the following dummy R script:

#!/usr/bin/env Rscript

args = commandArgs(trailingOnly = TRUE)

result = sum(as.numeric(args))

cat(result, '\n')

And I have created the following CWL specification:

- id: "#sumcmd"
  class: CommandLineTool
  inputs:
    - id: "#a"
      type: int
      inputBinding: {}
    - id: "#b"
      type: int
      inputBinding: {}
    - id: "#output-file"
      type: string
  outputs:
    - id: "#filename"
      type: File
      outputBinding: 
        glob: $(inputs['output-file'])
  baseCommand: sum.r
  stdout: $(inputs['output-file'])

If I try to run the tool, it seems fine at first:

vagrant@debian-jessie:~$ cwl-runner sum.cwl#sumcmd -h
/usr/local/bin/cwl-runner 1.0.20160222205901
usage: sum.cwl#sumcmd [-h] -a A -b B --output-file OUTPUT_FILE [job_order]

positional arguments:
  job_order             Job input json file

optional arguments:
  -h, --help            show this help message and exit
  -a A
  -b B
  --output-file OUTPUT_FILE

But when I try to run it with actual arguments:

vagrant@debian-jessie:~$ cwl-runner sum.cwl#sumcmd -a 50 -b 49 
--output-file foobar.txt
/usr/local/bin/cwl-runner 1.0.20160222205901
Got workflow error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cwltool/main.py", line 160, 
in single_job_executor
    for r in jobiter:
  File "/usr/local/lib/python2.7/dist-packages/cwltool/draft2tool.py", line 
131, in job
    j.stdout = builder.do_eval(self.tool["stdout"])
  File "/usr/local/lib/python2.7/dist-packages/cwltool/builder.py", line 164
, in do_eval
    context=context, pull_image=pull_image)
  File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py", line 
135, in do_eval
    return sandboxjs.interpolate(ex, jshead(r.get("expressionLib", []), 
rootvars))
  File "/usr/local/lib/python2.7/dist-packages/cwltool/sandboxjs.py", line 
131, in interpolate
    e = execjs(scan[w[0]+1:w[1]], jslib)
  File "/usr/local/lib/python2.7/dist-packages/cwltool/sandboxjs.py", line 
20, in execjs
    stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
Workflow error:
  [Errno 2] No such file or directory

At first I thought this was on me, but then I cloned the example workflows and tools repository, and tried to execute one of the included tools.

vagrant@debian-jessie:~$ cwl-runner workflows/tools/linux-sort.cwl --input 
kk.txt --output kk2.txt --key 1
/usr/local/bin/cwl-runner 1.0.20160222205901
Got workflow error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cwltool/main.py", line 160, 
in single_job_executor
    for r in jobiter:
  File "/usr/local/lib/python2.7/dist-packages/cwltool/draft2tool.py", line 
131, in job
    j.stdout = builder.do_eval(self.tool["stdout"])
  File "/usr/local/lib/python2.7/dist-packages/cwltool/builder.py", line 164
, in do_eval
    context=context, pull_image=pull_image)
  File "/usr/local/lib/python2.7/dist-packages/cwltool/expression.py", line 
135, in do_eval
    return sandboxjs.interpolate(ex, jshead(r.get("expressionLib", []), 
rootvars))
  File "/usr/local/lib/python2.7/dist-packages/cwltool/sandboxjs.py", line 
131, in interpolate
    e = execjs(scan[w[0]+1:w[1]], jslib)
  File "/usr/local/lib/python2.7/dist-packages/cwltool/sandboxjs.py", line 
20, in execjs
    stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
Workflow error:
  [Errno 2] No such file or directory

Problem is, I do not have any idea of what is exactly causing the error. I can see that is related to not finding a file or a directory, but I am not exactly sure about where to start looking for, because I think I do not understand what the cwl-runner tool is actually doing.

Any help or hint would be much appreciated.

Regards, Gustavo.

[tetron]

Hi @Keyeoh,

The immediate bug is that cwltool wants Node.js to interpret Javascript expressions, and it's unable to execute it, and that failure is not handled gracefully leading to backtraces. As of 6ce6370 you should get a more reasonable error message explaining what the problem is.

However, there's also a problem in your example, which is that you are using draft-3 expression syntax without providing cwlVersion: cwl:draft-3. Unfortunately cwltool lets you get away with but it's not valid, you should update your tool description:

cwlVersion: cwl:draft-3
$graph:
- id: "#sumcmd"
  class: CommandLineTool
  inputs:
    - id: "#a"
      type: int
      inputBinding: {}
    - id: "#b"
      type: int
      inputBinding: {}
    - id: "#output-file"
      type: string
  outputs:
    - id: "#filename"
      type: File
      outputBinding: 
        glob: $(inputs['output-file'])
  baseCommand: sum.r
  stdout: $(inputs['output-file'])

Or to write it more idiomatically:

class: CommandLineTool
cwlVersion: cwl:draft-3
inputs:
  - id: a
    type: int
    inputBinding: {}
  - id: b
    type: int
    inputBinding: {}
  - id: output-file
    type: string
outputs:
  - id: filename
    type: File
    outputBinding:
      glob: $(inputs['output-file'])
baseCommand: sum.r
stdout: $(inputs['output-file'])

[Keyeoh]

Thank you very much, @tetron, for your quick answer. Changed the cwl definition file to the second one you provide (the more idiomatic one) and it ran smoothly. Before closing the issue, could I ask you two small questions related to it?

• Why is the second one more "idiomatic". I wrote the first version just by following the examples in the repository. Are the examples outdated?
• Following the second approach, how can I define several objects in the same CWL file? There's no id tag in there.

Forgive me in advance if my questions are dumb, but I am a completely newbie in which refers to Javascript, Node, advanced JSON and related stuff..

[manu-chroma]

@mr-c can we close?

[mr-c]

@manu-chroma We should answer @Keyeoh's questions first