yuch7 / cwlexec

A new open source tool to run CWL workflows on LSF
Other
36 stars 8 forks source link

cwlexec fails to fail when a required input is not provided. #22

Closed drjrm3 closed 6 years ago

drjrm3 commented 6 years ago

A simple workflow which requires a string runs even when no input is provided. cwltool fails to run on the same example as expected.

[jmichael(BASH)@nodecn011]: cwlexec foo.cwl 
[15:51:20.864] INFO  - Workflow ID: ca52aa98-d283-4610-afde-b56a6b8e1ad9
[15:51:20.865] INFO  - Name: foo
[15:51:20.865] INFO  - Description file path: /research/rgs01/home/clusterHome/jmichael/cwlexec_bugs/non-optional-inputs/foo.cwl
[15:51:20.865] INFO  - Output directory: /home/jmichael/cwl-workdir/ca52aa98-d283-4610-afde-b56a6b8e1ad9
[15:51:20.865] INFO  - Work directory: /home/jmichael/cwl-workdir/ca52aa98-d283-4610-afde-b56a6b8e1ad9
[15:51:20.865] INFO  - Workflow "foo" started to execute.
[15:51:20.870] INFO  - Started job (foo) with
bsub \
-cwd \
/home/jmichael/cwl-workdir/ca52aa98-d283-4610-afde-b56a6b8e1ad9 \
-o \
%J_out \
-e \
%J_err \
-env \
TMPDIR=/home/jmichael/cwl-workdir/ca52aa98-d283-4610-afde-b56a6b8e1ad9 \
echo
[15:51:20.993] INFO  - Job (foo) was submitted. Job <61886769> is submitted to queue <normal>.
[15:51:21.009] INFO  - Started to wait for jobs by
bwait \
-w \
done(61886769)
[15:51:25.188] INFO  - The job (foo) <61886769> is done with stdout from LSF:

{ }
[jmichael(BASH)@nodecn011]: echo $?
0
[jmichael(BASH)@nodecn011]: cat foo.cwl 
#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool

baseCommand: echo

inputs:
  foo:
    type: string
    inputBinding:
      position: 1

outputs: []
[jmichael(BASH)@nodecn011]: cwltool foo.cwl 
/hpcf/apps/python/install/3.5.2/bin/cwltool 1.0.20180525185854
Resolved 'foo.cwl' to 'file:///research/rgs01/home/clusterHome/jmichael/cwlexec_bugs/non-optional-inputs/foo.cwl'
usage: foo.cwl [-h] --foo FOO [job_order]
foo.cwl: error: argument --foo is required
[jmichael(BASH)@nodecn011]: cwlexec --version
0.2.0
[jmichael(BASH)@nodecn011]: 
drjrm3 commented 6 years ago

Please note that I came across this error when I was trying to create a minimal example for a request to have more informative output when an input file is not provided. We have a more complex workflow for which I did not provide input and I got the following output so I think the error is caused by not providing an input.

a) I'm not sure why this output is generated on my complex example but not the simple example above. b) If the below output is, indeed, caused by simply not providing an input .yml then I would request that a more clear explanation be given as to the failure reason.

Error: Failed to run workflow, java.lang.NullPointerException
    at com.ibm.spectrumcomputing.cwl.exec.util.evaluator.InputsEvaluator.evalCWLFileArray(InputsEvaluator.java:125)
    at com.ibm.spectrumcomputing.cwl.exec.util.evaluator.InputsEvaluator.evalSecondaryFiles(InputsEvaluator.java:79)
    at com.ibm.spectrumcomputing.cwl.exec.util.evaluator.InputsEvaluator.eval(InputsEvaluator.java:59)
    at com.ibm.spectrumcomputing.cwl.exec.service.CWLInstanceService.buildMainInstance(CWLInstanceService.java:269)
    at com.ibm.spectrumcomputing.cwl.exec.service.CWLInstanceService.createMainInstance(CWLInstanceService.java:186)
    at com.ibm.spectrumcomputing.cwl.exec.service.CWLExecService.submit(CWLExecService.java:105)
    at com.ibm.spectrumcomputing.cwl.exec.CWLExecLauncher.runCommand(CWLExecLauncher.java:332)
    at com.ibm.spectrumcomputing.cwl.exec.CWLExecLauncher.launchCWLExec(CWLExecLauncher.java:104)
    at com.ibm.spectrumcomputing.cwl.Application.main(Application.java:33)
skeeey commented 6 years ago

Fixed at db9c2d6, if the required argument is not provided, the exit code will be 252 and the message will be like: The argument [foo] for step [required_args] is required.