Examples do not run as written

[prismofeverything]

I am trying to run the example "hello.cwl" as given, and I get the following error:

Tool file contains graph of multiple objects, must specify one of #echocmd, #main

Has cwl-runner changed to disallow multiple definitions per file? And if so, what is the idiomatic way to run this now?

Thanks!

[prismofeverything]

By looking at other examples I have determined you must give the id to cwl-runner on invocation. So this works:

$ cwl-runner test/hello.cwl#main
/usr/local/bin/cwl-runner 1.0.20160310140736
[job step0] /var/folders/2l/0wpdpqws4jvg9lqjwrhvdcl8_c3ksp/T/tmprCyEBm$ echo 'Hello World' > /var/folders/2l/0wpdpqws4jvg9lqjwrhvdcl8_c3ksp/T/tmprCyEBm/messageout.txt
[step step0] completion status is success
[workflow main] outdir is /Users/spanglry/Code/funnel
Final process status is success
{
    "output": {
        "path": "/Users/spanglry/Code/funnel/messageout.txt", 
        "checksum": "sha1$648a6a6ffffdaa0badb23b8baf90b6168dd16b3a", 
        "class": "File", 
        "size": 12
    }
}

This should probably be updated in the README, since it is the first example a new user will try!

However, the next example, "hello-params.cwl", fails in a different way:

$ cwl-runner test/hello-param.cwl#main test/hello-param.json 
/usr/local/bin/cwl-runner 1.0.20160310140736
Unexpected exception
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/cwltool/workflow.py", line 472, in job
    **kwargs):
  File "/usr/local/lib/python2.7/site-packages/cwltool/draft2tool.py", line 158, in job
    j.stdout = builder.do_eval(self.tool["stdout"])
  File "/usr/local/lib/python2.7/site-packages/cwltool/builder.py", line 164, in do_eval
    context=context, pull_image=pull_image)
  File "/usr/local/lib/python2.7/site-packages/cwltool/expression.py", line 135, in do_eval
    return sandboxjs.interpolate(ex, jshead(r.get("expressionLib", []), rootvars))
  File "/usr/local/lib/python2.7/site-packages/cwltool/sandboxjs.py", line 137, in interpolate
    e = execjs(scan[w[0]+1:w[1]], jslib)
  File "/usr/local/lib/python2.7/site-packages/cwltool/sandboxjs.py", line 47, in execjs
    raise JavascriptException("Returncode was: %s\nscript was: %s\nstdout was: '%s'\nstderr was: '%s'\n" % (nodejs.returncode, script, stdoutdata, stderrdata))
JavascriptException: Returncode was: 2
script was: console.log(JSON.stringify(require("vm").runInNewContext("\"use strict\";var inputs = {\"echo-in-message\": \"Hello, CWL !\\nHello World !\", \"echo-in-outputfile\": \"useroutput.txt\"};\nvar self = null;\nvar runtime = {\"outdirSize\": 1024, \"ram\": 1024, \"tmpdirSize\": 1024, \"cores\": 1, \"tmpdir\": \"/var/folders/2l/0wpdpqws4jvg9lqjwrhvdcl8_c3ksp/T/tmpLYxo8P\", \"outdir\": \"/var/folders/2l/0wpdpqws4jvg9lqjwrhvdcl8_c3ksp/T/tmpRa9EEa\"};\n(function(){return ((inputs['echo-in-outputfile']));})()", {})));

stdout was: ''
stderr was: 'Unable to find image 'node:slim' locally
slim: Pulling from library/node
fdd5d7827f33: Pulling fs layer
a3ed95caeb02: Pulling fs layer
0f35d0fe50cc: Pulling fs layer
b6c4c5059134: Pulling fs layer
580933eaf3e0: Pulling fs layer
b6c4c5059134: Waiting
580933eaf3e0: Waiting
a3ed95caeb02: Verifying Checksum
a3ed95caeb02: Download complete
b6c4c5059134: Verifying Checksum
b6c4c5059134: Download complete
'

Workflow error, try again with --debug for more information:
  Returncode was: 2
script was: console.log(JSON.stringify(require("vm").runInNewContext("\"use strict\";var inputs = {\"echo-in-message\": \"Hello, CWL !\\nHello World !\", \"echo-in-outputfile\": \"useroutput.txt\"};\nvar self = null;\nvar runtime = {\"outdirSize\": 1024, \"ram\": 1024, \"tmpdirSize\": 1024, \"cores\": 1, \"tmpdir\": \"/var/folders/2l/0wpdpqws4jvg9lqjwrhvdcl8_c3ksp/T/tmpLYxo8P\", \"outdir\": \"/var/folders/2l/0wpdpqws4jvg9lqjwrhvdcl8_c3ksp/T/tmpRa9EEa\"};\n(function(){return ((inputs['echo-in-outputfile']));})()", {})));

stdout was: ''
stderr was: 'Unable to find image 'node:slim' locally
slim: Pulling from library/node
fdd5d7827f33: Pulling fs layer
a3ed95caeb02: Pulling fs layer
0f35d0fe50cc: Pulling fs layer
b6c4c5059134: Pulling fs layer
580933eaf3e0: Pulling fs layer
b6c4c5059134: Waiting
580933eaf3e0: Waiting
a3ed95caeb02: Verifying Checksum
a3ed95caeb02: Download complete
b6c4c5059134: Verifying Checksum
b6c4c5059134: Download complete
'

Looks like it is trying to find a node:slim container and failing? Any ideas?

Thanks!

[prismofeverything]

I resolved this issue by pulling down the node:slim container first, then running the example.

[sb10]

This is not really resolved - the help text is still wrong!

This was indeed the first thing I tried as a new user, and not being able to get the "hello world" example working isn't a great first experience. Even reading your explanation I have no idea what the problem is and why adding #main is the solution.

[mr-c]

@sb10 Thank you for sharing your experience, I appreciate it. I agree this wasn't a good first user experience.

A CWL file can contain a single document or a graph of multiple documents. In the later instance a particular document must be specified as the entrypoint. This is rather advanced usage and not suitable for an introduction to CWL.

I've refactored hello.cwl to be simpler and to not need javascript in https://github.com/common-workflow-language/workflows/pull/124 Does that work better for you?

[sb10]

Thanks, if it runs following the existing help text that would be an improvement.

Though I question why an entry point must be specified? Why wouldn't a cwl-runner just pick the object with class "Workflow" and only complain if there were more than 1 in a cwl file?

Comparing your new hello.cwl to the old one, to be honest I prefer the old one, since it seems more natural and composable to me to specify steps independently of workflows.

[buchanae]

Same poor user experience for me. I ran this

cwltool cwl-workflows/workflows/hello/hello.cwl
/Users/buchanae/projects/bunny/venv/bin/cwltool 1.0.20170213175853
Resolved 'cwl-workflows/workflows/hello/hello.cwl' to 'file:///Users/buchanae/projects/bunny/cwl-workflows/workflows/hello/hello.cwl'
Tool definition failed initialization:
Tool file contains graph of multiple objects, must specify one of #echocmd, #main

and my first reaction was that the example is out of date. Luckily Google led me to this issue. In my opinion cwltool should fail with a better error, but I also think requiring an ID on the command line is strange. In my opinion, a workflow should just be a file, not a file + entrypoint ID.

[cnjr2]

I have build a quite large Workflow with nested subworkflows and relying on several CommandLineTools. In order to store everything in one place I have cwltool --packed it and it stopped running. I also tried to --validate the packed Workflow and it passed fine.

After reading this thread, it makes perfect sense to me why I have to specify the main workflow with packed_workflow.cwl#main when running the packed version.

Very happy now :smile:

Thanks!

[mr-c]

cwltool has defaulted to running the #main / main entry and in v1.1 we are codifying this behavior