At NREL we saw slurm fall over on startup. Hypothesis was that slurmctld startup was taking a while (due slurm.conf taking ~4s to read) but the unit returns immediately. Hence slurmds started and couldn't contact slurmctld, so went down.
Fix checks that slurmctld port is open before exiting the restart handler. 10s delay is to wait for slurmctld to go down. Ran OK on NREL.
At NREL we saw slurm fall over on startup. Hypothesis was that slurmctld startup was taking a while (due slurm.conf taking ~4s to read) but the unit returns immediately. Hence slurmds started and couldn't contact slurmctld, so went down.
Fix checks that slurmctld port is open before exiting the restart handler. 10s delay is to wait for slurmctld to go down. Ran OK on NREL.