search

SAME-Project / same-project

https://sameproject.ml/
Apache License 2.0
20 stars 8 forks source link

Support multistep pipelines in azure functions backend. #173

Closed Bubblyworld closed 2 years ago

Bubblyworld commented 2 years ago

The way to do this, I think:

  1. Add fields to the initiator JSON payload containing stuff like same.yaml, steps, requirements.txt, etc. So basically include everything.
  2. Change the executor to accept context blobs which contain serialised global namespaces (like we do for kubeflow/aml)
  3. Change orchestrator to fan-out each step and thread the contexts through.