nextflow config for CSD3

[tavareshugo]

Look at things like:

• executor options like queueSize, submitRateLimit, etc. (check @RaqManzano config)
• specific options for process_high_memory, etc. to submit to the most efficient queues (himem or regular).
• cache options for conda and singularity

[tavareshugo]

Some instructions that could be included in these materials.

In your ~/.nextflow/config file, set something like this:

process {
  executor = 'slurm'
  clusterOptions = '-A YOUR-SL-ACCOUNT -p icelake'
}

That will ensure that Nextflow submits jobs to SLURM, rather than run on the login node or in an interactive session. My usual procedure is:

• Login to the HPC and make a note of which node I was put in. For example, right now I'm on login-p-4, so write that down.
• Launch a persistent terminal using tmux (see here for tutorial on tmux) - this will make sure that I can logout of the HPC and then come back to my tmux terminal as if I've never left.
• Launch my nextflow pipeline. Nextflow should start launching jobs to SLURM.
• Detach from my tmux terminal and logout of the HPC.
• Go do other things...
• Once I'm ready to check back on progress, ssh into the HPC, but this time making sure you login to the same node as before - this is critical! In my example, I would do ssh hm533@login-p-4.hpc.cam.ac.uk
• Then attach myself back to my tmux session, the default session is callled "0", so tmux attach -t 0 (or do tmux ls to see all the sessions running on that node).
• Check what's going on with my Nextflow.