Open AlaaALatif opened 6 months ago
This is great!! Very excited to have the separated MQC outs and I think it's good to reduce the number of jobs for slurm. Such a bummer we can't easily set maximum cpus for slurm in nextflow, but I think what you have is a good call. Only comment is that we may want to consider setting up a profile that takes more time but fewer cpus? Not part of this PR of course, but something for the future.
key modifications to the pipeline:
executor.cpus
parameter inconfig/base.config
to be able to control maximum allowable number of CPU usage when running underlocal
profile i.e. running the pipeline on a local machine or on a developmental (dev) nodeexecutor.queueSize
parameter inconfig/base.config
to be able to control maximum allowable number of jobs to run when running the pipeline on a cluster via slurmmax_cpus_per_job
parameter inconfig/parameters
to be able to control maximum allowable number of CPU usage per job when running the pipeline on a cluster via slurmcall_snps
parameter inconfig/parameters.config
to be able to control whether or not SNP calling process steps need to be carried out. This was requested by the Genomics core and makes sense since these steps have considerably high computational requirements