Development notes

[tavareshugo]

Titles:

• Managing bioinformatic software and pipelines
• How to do bioinformatics like a professional bioinformatician
• Reproducible and scalable bioinformatics for researchers

Prerequisite:

• Unix
• HPC
• Knowledge of some bioinformatic application (RNA-seq, ChIP-seq, Bacterial, Viral, variant calling)

Outline:

• (30m) General intro - where we give a high-level view of all these concepts:
  • Software environments
  • Containers
  • Pipelines
• (1h30) Software (mamba)
  • Similar to the content on current HPC course
  • How to create and environment
  • How to install softawre on that environment
  • How to activate and run software
  • Concept of channels
  • Exercise: install FastQC and run your samples through
• (30m) Containers (singularity) - similar to HPC course
  • Introduce the concept of running things in a container
  • singularity pull and singularity run
  • Exercise: same as the conda exercise but using singularity
• (4.5h) Pipelines (nextflow nf-core focused)
  • What is a pipeline and what is it for? Why would you use it? Why you wouldn't want to use bash-based pipelines for example.
  • Conceptually separate Nextflow and nf-core
  • nf-core project
  • nf-core documentation
  • nextflow config - for running locally or on a server, use conda/singularity, etc.
  • set up the config locally
  • introduce a simple pipeline (e.g. rnaqc, bacQC)
  • for exercises we can have a collection of datasets from different applications for people to pick-and-choose (RNA-seq, ChIP-seq, bacterial, viral)
  • reference genome/annotation management - we should mention something about this. Advise downloading reference/annotation and using --fasta and --gff options - ensure reproducibility and uses latest version (or whichever version you wish to use). Discourage using igenomes (and why we don't recommend it). Use the reference that matches your downstream needs, e.g. using dbsnp or something like that.

Useful resources:

• nf-core training

[tavareshugo]

Initial draft of the timetable (names is to indicate who is doing which slides):

09:45 - 10:15 Introduction to reproducible and scalable bioinformatics analysis - Adam 10:15 - 11:00 Using package managers (slides + practical) - Adam 11:00 - 11:15 Break 11:15 - 12:00 Singularity (slides + practical) - Hugo 12:00 - 12:30 Nextflow (slides) - Raquel 12:30 - 13:30 Lunch 13:30 - 14:30 Nf-core demo and practical - Raquel 14:30 - 15:30 Configuring nextflow on HPC - Hugo 16:00 - 17:00 Q&A

[tavareshugo]

On the live workshop:

09:30 - 09:45 - Welcome 09:45 - 10:10 - Introduction 10:10 - 10:50 - Mamba 10:50 - 11:30 - Mamba Exercise 11:30 - 12:00 - Singularity slides 12:00 - 12:30 - Singularity exercise and wrap-up 12:30 - 13:30 - Lunch 13:30 - 14:15 - Nextflow slides 14:15 - 15:30 - Nextflow exercises (this could be reduced, see #27) 15:30 - 16:00 - Nextflow exercise solution (took a bit too long - don't need to go through the samplesheet python script in such detail) 16:00 - 16:30 - Nextflow on a HPC slides and tmux demo 16:30 - 17:30 - HPC exercise and wrap-up