Runnning multiple instance of workflow on slurm causes Denial-of-Service

snakemake / snakemake

This is the development home of the workflow management system Snakemake. For general information, see

https://snakemake.github.io

MIT License

2.25k stars 545 forks source link

Runnning multiple instance of workflow on slurm causes Denial-of-Service #2899

Closed shashwatsahay closed 3 months ago

shashwatsahay commented 4 months ago

Snakemake version 7.32.4

Running multiple instance of snakemake on slurm using --slurm hangs the workflow for hours with no respite

I have to run a workflow on 1000 samples individually runnnig it with slurm causes ddos on the slurm controller is there a workaround for this

johanneskoester commented 3 months ago

You can restrict the number of jobs in the queue via --jobs. The number depends on what your cluster is capable to handle. Also consider using Snakemake >=8, where the cli changes slightly.