carpentries-incubator / workflows-nextflow

Workflow management with Nextflow and nf-core
https://carpentries-incubator.github.io/workflows-nextflow/
Other
18 stars 29 forks source link

Engineering workflows to apply processes to a second/nth set of inputs seems to cause confusion. #59

Closed mahesh-panchal closed 2 months ago

mahesh-panchal commented 2 years ago

Nextflow does not allow processes/ subworkflows to be reused with the same name. This causes troubles engineering workflows when a learner wants to apply the same process to another output channel. e.g. https://twitter.com/yokofakun/status/1466101898625363968?t=Iw-opIManrMXGjKLLlmhxw&s=03 (slightly different case)

This use case perhaps needs to be more explicit in the possible solutions.

  1. Often we think of a workflow as a series of steps, e.g. read QC, trimming, post-trimming QC, resulting in trying:
    workflow {
    FASTQC( read_ch )
    TRIM_READS( read_ch )
    FASTQC( TRIM_READS.out.trimmed_reads )
    }

    It's not often intuitive that one can swap the position of lines 2 and 3, and then use a channel operator like mix to merge channels for processing.

    workflow {
    TRIM_READS( read_ch )
    FASTQC( read_ch.mix( TRIM_READS.out.trimmed_reads) )
    }
  2. The alternative is using an alias and separating the process/subworkflow into a module. This option also enhances readability and helps follow a writers natural mental workflow flow, and allows for additional configuration using process selectors.