Closed psteinb closed 4 years ago
I'm not sure it will work in this container since Slurm is compiled with --enable-front-end
. On frontend systems, I believe the only TaskPlugin
available is task/none
.
Ok, I'll have a look at this flag. For me, I am running on a 40 core machine. I'd be fine with spinning up containers for 4 compute nodes and a front-end node and potentially a slurmdb node. I am not an expert on slurm, nor an admin, but IIRC that is all that is needed. Once could then use docker compose to spin up multiple containers and somehow do the networking between them. For your use case, this might be overdoing it. For the use case of simulating a slurm based cluster, it would be the best. Then, the cgroups configuration would be possible, I guess.
A lot of guess work plus the validation with reality. What do you think?
I'll try compiling a different way to see what options I have for cgroup configuration.
Super cool, looking forward to it. P
Have you seen this project? It uses docker-compose
for spinning up a slurm cluster locally and the networking and volumes are shared across nodes. I haven't quite kept it up to date, but it would be easy to update the versions. Is this similar to what you are trying to accomplish here?
Err, yes I did see it ... but I didn't get anywhere to be honest. This project appeared more maintained and supported the slurm version I needed out of the box. So I opted for it. I am happy to try the other one - but my limitation is that all nodes run on the same host.
I am wondering if this setup could be used to simulate a slurm configuration using the
task/affinity
ortask/cgroups
mode forTaskPlugin
. How do you feel @giovtorres ?