search

uwefladrich / scriptengine-tasks-hpc

ScriptEngine Task set for HPC systems
GNU General Public License v3.0
2 stars 2 forks source link

Add support for SLURM hetereogeneous jobs #5

Closed uwefladrich closed 3 years ago

uwefladrich commented 3 years ago

hpc.slurm.sbatch should support SLURM hetereogeneous jobs. This means that colon-separated options and arguments need to be handled by the task.

uwefladrich commented 3 years ago

The suggested syntax would be:

hpc.slurm.sbatch:
    <general options>
    hetjob_spec:
        <list of hetjob opts+args>

for example:

hpc.slurm.sbatch:
    account: foo
    time: 100
    output: foo.out
    hetjob_spec:
        - nodes: 1
        - nodes: 2
        - nodes: 1

which would translate to

sbatch \
  --account foo --time 100 --output foo.out \
  --nodes 1 : --nodes 2 : --nodes 1

and queue a heterogeneous job with three hetjob components on one, two, and one nodes, respectively.