search

d-krupke / slurminade

A decorator-based slurm runner.
https://slurminade.readthedocs.io
MIT License
12 stars 4 forks source link

Fixes d-krupke/slurminade#8 #9

Closed p-blumenberg closed 1 year ago

p-blumenberg commented 1 year ago

Added call to dispatch_guard in Dispatcher. Added tests for dispatch limit with and without batches.

p-blumenberg commented 1 year ago

Should calls to sbatch and srun also be counted?

d-krupke commented 1 year ago

Yes. They also create slurm tasks that in case of a bug could overload slurm.

p-blumenberg commented 1 year ago

We call dispatch_guard from every Dispatcher subclass now because there was no straightforward way to do this in the parent class without breaking anything. Let me know if you want me to implement a more maintainable solution.

d-krupke commented 1 year ago

Looks good! I don't see a much better way to do it right now. It is not beautiful but as long as it is this simple, I don't think we need a more maintainable solution. Better to continue on the problem with too long calls with batches (the thing with the temp file solution). :thinking: Could you please bump up the version number (last number because it is a bugfix) and add an entry to the changelog? I will then merge it and trigger the distribution.