Improving nextflow support

[grst]

Hi @nvuillam,

I'm reaching out as a contributor to the nf-core project. We develop state-of-the art Bioinformatics pipelines in nextflow. We are highly interested in linting (or rather autoformatting) our pipelines, and I was thrilled to find out that npm-groovy-lint already can handle the Groovy parts in .nf files.

It would be amazing if npm-groovy-lint could be improved on the nextflow parts (see some non-exhaustive examples in the comments in the code below). We might be able to work on it during one of our next hackathons.

How complicated do you think it would be to improve the nextflow support? And if we were ready to tackle it, could you provide us some guidance where to start?

Best regards, Gregor

---

include { initOptions; saveFiles; getSoftwareName } from './functions' 

params.options = [:]
def options    = initOptions(params.options)

// -> automatically and consitently wrap channel transformations , should they get too long
someChannel = someOtherChannel. splitCsv(header:true, sep:',').map { 
    check_samplesheet_paths(it) 
}

process FASTQC {
    tag "$meta.id"
    label 'process_medium'

// -> automatically wrap lines; indent 'mode' and 'saveAs', which are options for publishDir
    publishDir "${params.outdir}",
    mode: params.publish_dir_mode,
    saveAs: { filename -> saveFiles(filename:filename, options:params.options, publish_dir:getSoftwareName(task.process), publish_id:meta.id) }

    conda (params.enable_conda ? 'bioconda::fastqc=0.11.9' : null)

// -> wrap lines, indent if
    if (workflow.containerEngine == 'singularity' && !params.singularity_pull_docker_container) { container 'https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0' } else { container 'quay.io/biocontainers/fastqc:0.11.9--0' }

// -> consistently indent input and output channels
    input:
    tuple val(meta), path(reads)

    output:
        tuple val(meta), path('*.html'), emit: html
        tuple val(meta), path('*.zip'), emit: zip
        path  '*.version.txt'          , emit: version

    script:
    def software = getSoftwareName(task.process)
    def prefix   = options.suffix ? "${meta.id}.${options.suffix}" : "${meta.id}"
    if (meta.single_end) {
        """
        [ ! -f  ${prefix}.fastq.gz ] && ln -s $reads ${prefix}.fastq.gz
        fastqc $options.args --threads $task.cpus ${prefix}.fastq.gz
        fastqc --version | sed -e "s/FastQC v//g" > ${software}.version.txt
        """
    } else {
        """
        [ ! -f  ${prefix}_1.fastq.gz ] && ln -s ${reads[0]} ${prefix}_1.fastq.gz
        [ ! -f  ${prefix}_2.fastq.gz ] && ln -s ${reads[1]} ${prefix}_2.fastq.gz
        fastqc $options.args --threads $task.cpus ${prefix}_1.fastq.gz ${prefix}_2.fastq.gz
        fastqc --version | sed -e "s/FastQC v//g" > ${software}.version.txt
        """
    }
}

[nvuillam]

Hi Gregor :)

npm-groovy-lint relies on:

• CodeNarc for detecting rules
• npm-groovy-lint itself for formatting and auto-fixes

1. So if you want to implement new rules related to nextflow, you need to create new rules in CodeNarc

I don't know if they could be included in CodeNarc release or delivered as a CodeNarc plugin (@chrismair, the amazing CodeNarc maintainer, could answer this question if you post an issue there)

2. Then, if you want to manage auto-fixing of rules, it will be on npm-groovy-lint, in rules folder, but it requires CodeNarc returned errors, so this can be only after the step 1

For information, I added npm-groovy-lint in Mega-Linter, who lints natively .nf files, so if you need npm-groovy-lint seamless integration in your CI jobs, you can have a look there ;)

[grst]

Thanks @nvuillam! I created https://github.com/CodeNarc/CodeNarc/issues/575 to follow up.

[nvuillam]

@grst i saw the reply on CodeNarc If you want npm-groovy-lint to be delivered with your extra JAR containing nextflow rules, that won't be a problem :)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.