[@parallel][jobsets] bug fixes and refactor - Githubissues

Netflix / metaflow

:rocket: Build and manage real-life ML, AI, and data science projects with ease!

https://metaflow.org

Apache License 2.0

7.79k stars 737 forks source link

[@parallel][jobsets] bug fixes and refactor #1900

Open valayDave opened 1 week ago

valayDave commented 1 week ago

Abstraction over Jobset Spec
Create general abstraction and copy style of jobspec for native k8s jobs for jobsets
simplified the implementation.
similar pattern/trend to follow for argo
removed older implementation
Superseds #1854

Review commits individually to see core changes.

One commit has the entire refactor into a more decoupled implementation.
One commit has fixes to the core such as ID minting for control jobs.