This PR adds a new parameter to the experiments section. This parameter limits the maximum number of runs that slurm executes at once (extending the slurm array syntax that we already use: https://slurm.schedmd.com/job_array.html).
experiments:
- name: foo
args: ['foo.py']
stdout: out
max_simultaneous: 2
Use case: sometimes license limitations prevent us from running more than a specific number of experiments at once.
Limitation of this implementation: since this PR just extends the slurm array syntax, this setting only applies to a single launch. When launching multiple sets of experiments, the limit will be applied to each set individually.
This PR adds a new parameter to the
experimentssection. This parameter limits the maximum number of runs that slurm executes at once (extending the slurm array syntax that we already use: https://slurm.schedmd.com/job_array.html).Use case: sometimes license limitations prevent us from running more than a specific number of experiments at once.
Limitation of this implementation: since this PR just extends the slurm array syntax, this setting only applies to a single launch. When launching multiple sets of experiments, the limit will be applied to each set individually.