FaaSr / FaaSr-package

Function-as-a-Service package for R
https://faasr.io
Other
3 stars 7 forks source link

Support invocation of multiple concurrent actions #89

Open renatof opened 5 months ago

renatof commented 5 months ago

The goal here is to expand workflow configuration, trigger generation, and self-aborts to support static parallelism defined in the workflow across action invocations.

Workflow configuration: Allow users to express degree of parallelism in InvokeNext, e.g. "sum(10)" to invoke "sum" MaxRank=10 times

Trigger generation: Expand faasr_trigger() to parse the above, loop through to generate multiple triggers, and insert a unique integer "Rank" in the JSON payload for each trigger (e.g. Rank=1 .. Rank=10 in the example above)

User query interface: Expose a new faasr_rank() function that returns to the user both MaxRank and Rank (e.g. 10 and 1 in the example above)

Self-aborts: Expand faasr_abort_on_multiple_invocations() and the generation of .done file to account for multiple invocations; i.e. invocations with parallelism generate sum.1.done, sum.2.done as in the example above, and successor actions wait on all sum.X.done where X=1..MaxRank

spark0510 commented 5 months ago

A note: When triggering GitHub Actions, an action doesn't deliver json configuration to next action unlike other platforms. We may need to update the json in the repository. Another way to do is add 'input' to the yaml file to deal with 'Rank'.