Ideas for coding best-practice examples

[ewels]

Pipeline Scripts

• [ ] Process basics (input / output / script, channel types, output patterns, tag)
• [ ] Use of params (mention need for explicit defaults)
• [ ] Paired end inputs (FromFilePairs) and single-end (FromPath)
• [ ] Processes discriminating between PE and SE channels
• [ ] Process settings (cpus, memory, time etc)
• [ ] Clever restarting of failed jobs (errorStrategy, maxRetries etc)
• [ ] Publishing (publishDir, different modes, using params.outdir)
• [ ] Using the bin directory for bundling scripts
• [ ] R Scripts and R package installation

Config files

• [ ] Config file params, overwriting script params
• [ ] Config file process defaults (all processes)
• [ ] Config file process settings (single processes)
• [ ] Overwriting these defaults (eg. not loading MultiQC by using user config file)
• [ ] Nextflow profiles, loading different config files

Running on UPPMAX

• [ ] ClusterOptions (params.project, params.clusteroptions)
• [ ] Cluster queue type (process.queue = 'devcore')
• [ ] Pipeline checking that project is set if UPPMAX profile
• [ ] Preset genome paths in config (params.genome, creating channels from paths, using these channels in a way that they are not consumed)

Running outside UPPMAX

• [ ] Docker
• [ ] AWS
• [ ] Travis (mention memory limit

Bonus features

• [ ] Multiple reference index input options (building missing indexes if not supplied)
• [ ] Required structure for a pipeline to work with Nextflow GitHub support (main.nf etc)
• [ ] Consistency within NGI pipelines (naming NGI-xyz, readme template, versioning)

Common Processes

• [ ] FastQC
• [ ] TrimGalore! (with individual trimming params and grouped presets)
• [ ] MultiQC
• [ ] samtools sort / index / stats / flagstat
• [ ] Picard MarkDuplicates
• [ ] Build generic index / do generic alignment ..?

[ewels]

Oops, this got a bit longer than I intended.. But each bit doesn't need to be very long. Maybe different markdown files with each of these as a header? Then copy in a short code example if it's not very long.

Phil

[Hammarn]

I've tried starting on this several times now but I struggle to get anywhere since a lot of this seems redundant. Most points are covered pretty well by the official Nextflow documentation, example scripts or FAQ. I agreed that it would be nice to show how we tend to write scripts (with separate config files etc.) but a more generic help documentation seems a bit out of our scope.