It would be nice to have the ability to submit multiple task instances of a particular workflow to run in parallel. This is supported already if the workflow is configured to consume files in an input directory, but we should also allow this for workflows that don't accept inputs.
Use case
One potential use case here might be Monte Carlo simulations, with each starting from a unique random seed: such a workflow might not need any input data, but we would still like to be able to submit some large number of runs at once.
Implementation
Frontend
We can check in workflow.vue if the workflow accepts inputs — if not, append a spin button on the right side here:
Maybe called Iterations or Repetitions or Trials, defaulting to 1. We should only show this for immediate and delayed tasks (doesn't apply to triggered tasks, since no inputs, and might be too easily abused with repeating tasks).
Backend
We can use the TACC Launcher or SLURM job arrays to parallelize the task as we do for workflows with files-type inputs.
wpbonelli
commented
2 years ago
Overview
It would be nice to have the ability to submit multiple task instances of a particular workflow to run in parallel. This is supported already if the workflow is configured to consume files in an input directory, but we should also allow this for workflows that don't accept inputs.
Use case
One potential use case here might be Monte Carlo simulations, with each starting from a unique random seed: such a workflow might not need any input data, but we would still like to be able to submit some large number of runs at once.
Implementation
Frontend
We can check in
workflow.vueif the workflow accepts inputs — if not, append a spin button on the right side here:Maybe called
IterationsorRepetitionsorTrials, defaulting to 1. We should only show this for immediate and delayed tasks (doesn't apply to triggered tasks, since no inputs, and might be too easily abused with repeating tasks).Backend
We can use the TACC Launcher or SLURM job arrays to parallelize the task as we do for workflows with
files-type inputs.