search

yuch7 / cwlexec

A new open source tool to run CWL workflows on LSF
Other
36 stars 8 forks source link

Bad Scatter in subworkflow #33

Closed drjrm3 closed 5 years ago

drjrm3 commented 5 years ago

I have a CommandLineTool (foo.cwl), Workflow (inner.cwl) which calls foo.cwl, and another workflow (outter.cwl) which scatters over inner.cwl. Both foo.cwl and inner.cwl work as intended, but outter.cwl does not properly scatter over inner.cwl. That is, instead of scattering over the 'input_file' as it should, it simply calls 'foo' with all possible 'input_file's as input.

It should be:

foo --INPUT file1.txt foo --INPUT file2.txt

but instead, it invokes foo as

foo --INPUT file1.txt file2.txt

This may be related to issue 20 which I saw was recently moved to 'enhancement' rather than 'known issue'. In either case I'm hoping we will be able to scatter over subworkflows in this way as it will be very useful for our pipelines.

In the attached 'BadScatter.tar.gz' you will see that '01_foo.sh' gives the expected output, '02_inner.sh' gives the expected output, but '03_outer.sh' fails and that the invocation is a single call to foo with all possible input files, though the intent is for them to be scatter over as described above.

BadScatter.tar.gz

skeeey commented 5 years ago

@drjrm3, yes, this is related to #20. Currently, the cwlexec will ignore the sactter attribute in subflow and treat the subflow as a single flow to run.

As you know, we plan to fix this problem at #20, so I close this problem and trace this problem at #20

drjrm3 commented 5 years ago

Thank you for verifying this is the same issue. In the mean time I will try to use the workaround proposed for #20.