Open tomalrussell opened 3 years ago
Just to add few variations on the user story:
Hi @tomalrussell - as Rose mentioned on the DAFNI slack channel, we have work lined up to meet this along with some other requirements for parallel processing from @fmcclean and the OpenCLIM group. It's a big piece of work and so any more information on your particular use case is very much appreciated.
Captured in (internal) DAFNI ticket CORE-279
While duplication of an existing step in a workflow has not yet been added, you can now loop over a model step and change the timesteps and other parameter sets with all other parameters the same.
The only additional feature I can see here is the possibility of looping over a loop (I believe that is what would be needed for the final bullet point).
I've got a model to run over multiple steps, with mostly the same configuration, varying one or two inputs.
It would be handy to be able to duplicate a workflow step, then change the relevant values in the web user interface. This would reduce the amount of manual entry when working with a non-default set of inputs that should mostly stay the same between steps.
Another approach to the same problem would be a (possibly sequential, possibly parallel) iterator, which could take a series of values and apply them to model datasets // inputs. In the sequential version, e.g. running a model over multiple timesteps, it may be necessary to make outputs from one step in the iteration available in the next. In the parallel version, e.g. running a model over multiple scenarios or with parameters varying to conduct a sensitivity analysis, there may be no data dependency between each of the iteration steps, but there would likely be an aggregation step after the iterator (a "reduce" step in map-reduce terms).
Another approach would be an API or config-file interface to creating workflows, where repetitive confiugration could be scripted or copy-pasted and quickly edited by hand.