carpentries-incubator / workflows-nextflow

Workflow management with Nextflow and nf-core
https://carpentries-incubator.github.io/workflows-nextflow/
Other
21 stars 34 forks source link

Mental model confusion example for channels. #30

Closed mahesh-panchal closed 5 months ago

mahesh-panchal commented 3 years ago

I'm not sure where to record this yet, so I decided to store it as an issue here for now.

I was just demonstrating how to use Nextflow and I came across this difficulty one had in building their mental model. Might be useful for review later.

script:

#! /usr/bin/env nextflow

nextflow.enable.dsl = 2

workflow {

    channel.of(1, 2, 3, 4).set { input_ch }
    FOO(input_ch)

}

process FOO {

    input:
    val nums

    output:
    path "*.txt"

    script:
    """
    echo "$nums" > ${nums}.txt
    """
}

The confusion was why did we need both input_ch and nums in the script ( solution was explained by their colleague - peer instruction ). Their explanation: input_ch is a variable that has all the inputs, while nums only takes one value. nums is like an i in a bash for loop.

mahesh-panchal commented 2 years ago

I just managed to explain this with a pipes and marbles extraction that seemed to work nicely.

There is a distinction between the channel itself ( think of a pipe ), and what goes through it ( e.g. marbles - path objects/ files for instance )

The workflow block shows how to connect the "pipes", while the process input:/output: block tells you the type of "marble" it's expecting (e.g. blue marble = val type ). Each line in the input: corresponds to the order of the "pipes" in the workflow (include code with line numbers as demo).