search

martius-lab / cluster_utils

https://cluster-utils.readthedocs.io/stable/
Other
12 stars 0 forks source link

Implement cluster_requirements.concurrency_limit for Slurm #49

Closed luator closed 4 months ago

luator commented 11 months ago

In GitLab by @luator on Nov 27, 2023, 11:47

luator commented 10 months ago

This doesn't seem to be easily possible with Slurm. See https://stackoverflow.com/questions/42812425/limit-the-number-of-running-jobs-in-slurm Options are:

  1. Using job arrays. This doesn't fit well into how cluster_utils manages jobs at the moment, though.
  2. Set MaxJobs for the user, but a quick test I made suggests that this requires root permissions, so not a viable solution.
  3. Manage everything on the level of cluster_utils (i.e. cluster-system-agnostic). This would probably be a bigger effort.

@mseitzer what do you think how important this feature is? Assuming it is just nice-to-have but not crucial, I would close this issue as wont-fix due to the effort that would be needed.

By Felix Widmaier on 2024-01-16T11:27:24 (imported from GitLab)

luator commented 4 months ago

Closing as wont-fix now. Feel free to re-open in case the feature is really needed.