Is your feature request related to a problem? Please describe.
Our current Slurm scripts are a combination of 2 bash scripts that might be difficult to understand and customize in other user environments since it has some assumptions baked in (enroot/pyxis for containers), specific cluster setup etc.
There have been advancements made to Dask job queue's slurm runner which should make it easier to launch multi-node jobs in a similar environment to what we do.
In theory it should make it easier to launch MN slurm jobs with all the setup info shared as a part of the runner API.
It could be worth exploring if this makes our multi-node slurm setup a bit simpler.
Is your feature request related to a problem? Please describe. Our current Slurm scripts are a combination of 2 bash scripts that might be difficult to understand and customize in other user environments since it has some assumptions baked in (enroot/pyxis for containers), specific cluster setup etc. There have been advancements made to Dask job queue's slurm runner which should make it easier to launch multi-node jobs in a similar environment to what we do. In theory it should make it easier to launch MN slurm jobs with all the setup info shared as a part of the runner API.
It could be worth exploring if this makes our multi-node slurm setup a bit simpler.
Thanks @jacobtomlinson for the suggestion!