search

kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.5k stars 1.58k forks source link

[feature] Add support for dynamic DAGs #10713

Open anishasthana opened 2 months ago

anishasthana commented 2 months ago

Feature Area

/area backend

What feature would you like to see?

I would like to see support for dynamic dags

What is the use case or pain point?

There are some use-cases that may necessitate creation of a undetermined number of component from within a component.

For example:

  1. Have a training component running that has a variable number of checkpoints that may be created.
  2. For each checkpoint that is created, trigger a component to run for evaluation of the checkpoint
  3. The next component of the pipeline would wait until all "checkpoint evaluation" components have completed running, and do something.

Is there a workaround currently?

A somewhat hacky workaround using a queueing system as well as a pod monitoring events on the cluster.

cc @HumairAK

Love this idea? Give it a 👍.

github-actions[bot] commented 2 weeks ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

HumairAK commented 2 weeks ago

/remove-lifecycle stale