search

substratusai / runbooks

Finetune LLMs on K8s by using Runbooks
https://www.substratus.ai
Other
168 stars 14 forks source link

Understanding an object's provenance - should we track parents? #151

Open brandonjbjelland opened 1 year ago

brandonjbjelland commented 1 year ago

I wonder if it'd be valuable to have a common .status.parent field particularly for Dataset and Model. This is coming back to a need to understand a model's (and dataset's) lineage since we don't have a concept of pipelines (with a history) which would otherwise tell the story.

nstogner commented 1 year ago

The scenario where one object created another object is typically tracked with owner references in kubernetes: https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/

nstogner commented 1 year ago

Once we start enforcing immutability of .spec fields we will be able to trust the lineage specified there (i.e. .spec.trainingDataset in a Model).