Check if we can set a limit to resources consumed by nextflow

UCSF-DSCOLAB / data_processing_pipelines

A repository to store the existing pipelines to process the various CoLabs datasets

0 stars 1 forks source link

Check if we can set a limit to resources consumed by nextflow #43

Open amadeovezz opened 10 months ago

amadeovezz commented 10 months ago

We ideally would like to set a limit on the amount of resources used by a pipeline. For instance:

run entire pipeline with no more than 100GB of ram

erflynn commented 10 months ago

we can do max number of jobs -- e.g. set executor.queueSize = 60 for slurm for bulk https://www.nextflow.io/docs/latest/config.html It unfortunately looks like we cant set max cpus for SLURM