yuch7 / cwlexec

A new open source tool to run CWL workflows on LSF
Other
36 stars 8 forks source link

cwltool and cwlexec not scattering jobs in same way #4

Closed drkennetz closed 6 years ago

drkennetz commented 6 years ago

I have a simple workflow in which I have 2 inputs, one is type: File, and the other is an array of files. I want to run a command in which each of the files in the array are used against the single input file from input1. When I run cwltool it performs as expected. It runs in serial each file in the array against the single input1. When I run cwlexec it gives me no errors or message, and the job immediately terminates.

the tool is:


cwlVersion: v1.0
class: CommandLineTool

hints:
  SoftwareRequirement:
    packages:
      bedtools:
        version: [ "2.25.0" ]

inputs:
  outputGenomeCov:
    type: File
    inputBinding:
      position: 1
      prefix: -a

  regionsBedFile:
    type: File
    inputBinding:
      position: 2
      prefix: -b

  allPositions:
    type: string
    default: "-c"
    inputBinding:
      position: 3
      prefix: -c

outputs:
  allDepthOutput:
    type: File
    outputBinding: {glob: $(inputs.regionsBedFile.basename)_AtoB.txt}

stdout: $(inputs.regionsBedFile.basename)_AtoB.txt

baseCommand: [bedtools, intersect]

And the workflow is:


cwlVersion: v1.0
class: Workflow

requirements:
 - class: ScatterFeatureRequirement

inputs:
  outputGenomeCov: File
  regionsBedFile: File[]

outputs:
  intersectAB:
    type: File[]
    outputSource: intersect/allDepthOutput

steps:
  intersect:
    run: 2_bedtoolsIntersect.cwl
    scatter: regionsBedFile
    in:
      outputGenomeCov: outputGenomeCov
      regionsBedFile: regionsBedFile
    out: [allDepthOutput]

The .yml file:

outputGenomeCov:
  class: File
  path: /path/to/input.txt

regionsBedFile:
 - {class: File, path: /path/to/bedfile1.bed}
 - {class: File, path: /path/to/bedfile2.bed}
 - {class: File, path: /path/to/bedfile3.bed}

I have another workflow that has designated output files created from the program, but bedtools prints its outputs to stdout. The other workflow works well, but this fails. Thanks, Dennis

liuboxa commented 6 years ago

This issue has the same reason with #7