ICB-DCM / pyABC

distributed, likelihood-free inference
https://pyabc.rtfd.io
BSD 3-Clause "New" or "Revised" License
201 stars 44 forks source link

Job scheduler script in documentation can be improved by using Slurm job arrays #637

Open e-eight opened 4 months ago

e-eight commented 4 months ago

The job scheduler script for Slurm in the documentation uses a for loop to submit jobs:

#!/bin/bash

for i in {1..{n_jobs}}
do
  sbatch script_redis_worker.sh
done

However this is not necessarily optimal, particularly if n_jobs is large. It can bog down the HPC system. An alternative is to use Slurm job arrays. I recommend that the docs are updated to suggest Slurm job arrays as an alternative.

stephanmg commented 4 months ago

@e-eight thanks for the suggestion. we will update the docs accordingly.