Issue 16: more slurm versions

LourensVeen commented 2 years ago

Hi, and thanks for making and sharing this handy set-up. It's been very useful!

I've been doing some work on https://github.com/vecma-project/QCG-PilotJob, which uses an old fork of slurm-docker-cluster for its test setup. I wanted to test a broad range of Slurm versions, so I made some upgrades, and I figured I'd contribute them back here as well.

This PR upgrades to Rocky Linux 8, which seems to be the natural successor to the old CentOS. (I'm a Debian guy, so I hope I picked the right one.) It also fixes some issues I ran into while compiling different versions of Slurm. For me, this now works with the latest versions of 17.02, 17.11, 18.08, 19.05, 20.02, 20.11 and 21.08.

The final commit makes the Slurm version configurable in docker-compose.yaml, but that does mean that you always have to set it, so it may break things for people who have scripts calling docker-compose. You may want to cherry-pick the others rather than merging this wholesale.

This fixes #16 I think, CentOS 8 no longer being available.

giovtorres commented 2 years ago

Hi @LourensVeen, thank you for this contribution!

I was a big fan of CentOS for many years, and I'm still sad that we have to make this change. I'm ok with Rocky Linux, seems like the next logical succession for CentOS.

I like the support for the additional versions. We could use defaults in the docker-compose so that it does not fail for folks.

Something like this where you can provide defaults:

slurmdbd:
    image: slurm-docker-cluster:${SLURM_TAG:-slurm-21-08-6-1}
    build:
      context: .
      args: 
        SLURM_TAG: ${SLURM_TAG:-slurm-21-08-6-1:-slurm-slurm-21-08-6-1}

Then, you would docker-compose up to build and run the default version above, slurm-21-08-6-1, or you can override as follows: SLURM_TAG=slurm-21-08-6-1 docker-compose build && docker-compose up.

Thoughts?

Also, another tip to speed up the git clone, you can checkout just the branch you need instead of all since Slurm has git index is pretty big, e.g.

git clone -b ${SLURM_TAG} --single-branch --depth=1 https://github.com/SchedMD/slurm.git \

instead of this: https://github.com/giovtorres/slurm-docker-cluster/blob/3e6edfd8becbc97570eaee1484540cd2af2e56af/Dockerfile#L55

LourensVeen commented 2 years ago

Oh neat, I didn't know docker-compose variables could have default values. Yes, that would be much better.

Maybe there should be two variables, one for the git commit/branch/tag to build and the other one the version tag for the generated image? So you can build git branch slurm-21.08 and call it21.08-latest? I've done that now with SLURM_TAG and IMAGE_TAG.

Building takes a lot longer than downloading for me so I hadn't considered checking out only the branch, but it's a good idea to save some bandwidth. All those RPMs are bad enough :-).

Incidentally, the last commit, with your clone-the-branch only command, makes a slight change in that it now checks out ${SLURM_TAG} rather than tags/${SLURM_TAG}. I'd say that's a feature though, makes it more flexible, and backwards compatible too.

salanki commented 2 years ago

Chance to get this merged?

giovtorres / slurm-docker-cluster

Issue 16: more slurm versions #21