Let's say a user launches an experiment that involves, say, an inference job and an evaluation against human ground-truth. That experiment will launch 2 jobs. Subsequent runs of the experiment will lunch more iterations of those.
When the user gets the information of that experiment by its ID, this should include the list of jobs that the experiment ran.
Describe alternatives you've considered
A) Get /experiments/{id} returns a full list of all jobs, regardless of whether they were launched in the same iteration or not. This likely makes sense while we have not introduced an external tool to track experiments.
B) We adopt MLFlow (and others) abstraction of runs as an iteration of an experiment, meaning that get would return... a list of related runs? [Suggestion, this could be done once we outsource experiment tracking.]
Describe the solution you'd like
Let's say a user launches an experiment that involves, say, an inference job and an evaluation against human ground-truth. That experiment will launch 2 jobs. Subsequent runs of the experiment will lunch more iterations of those.
When the user gets the information of that experiment by its ID, this should include the list of jobs that the experiment ran.
Describe alternatives you've considered A) Get /experiments/{id} returns a full list of all jobs, regardless of whether they were launched in the same iteration or not. This likely makes sense while we have not introduced an external tool to track experiments. B) We adopt MLFlow (and others) abstraction of runs as an iteration of an experiment, meaning that get would return... a list of related runs? [Suggestion, this could be done once we outsource experiment tracking.]