search

snakemake / snakemake-executor-plugin-slurm

A Snakemake executor plugin for submitting jobs to a SLURM cluster
MIT License
15 stars 17 forks source link

Dynamic determination of queue #61

Open johanneskoester opened 6 months ago

johanneskoester commented 6 months ago

Would be great if we could specify a queue based on resources (like mem_mb, gpu, ...).

cmeesters commented 6 months ago

What do you mean by that? Could you elaborate, please?

johanneskoester commented 6 months ago

Let's chat about that in person, maybe this is also something for the main Snakemake source. I am not yet entirely clear about it.

cmeesters commented 2 months ago

@johanneskoester We have not addressed this — yet. Meanwhile, I see the slurm-gustave-roussy integrated into the catalogue. This is a bit unfortunate because it is a one-cluster only solution, and we loose maintainability or feature completeness, if we further support the proliferation of cluster plugins for one batch system. On the other hand, though, we strive to provide a convenient UI. One without the need to write complex profiles for every cluster.

An idea presumably similar to yours has been discussed (see #106). I elaborated, that this is nice to have, but difficult to implement because every cluster is different and SLURM allows coding additional plugins, which in turn allow having similar features addressed by different user syntax. If you've got time, either next week or (better) after my holidays.