mozilla-ai / lumigator

Source code for Mozilla.ai's Lumigator platform
https://mozilla-ai.github.io/lumigator/
Apache License 2.0
63 stars 7 forks source link

Getting an experiment_id includes the jobs related to that experiment #408

Open ividal opened 1 day ago

ividal commented 1 day ago

Describe the solution you'd like

Let's say a user launches an experiment that involves, say, an inference job and an evaluation against human ground-truth. That experiment will launch 2 jobs. Subsequent runs of the experiment will lunch more iterations of those.

When the user gets the information of that experiment by its ID, this should include the list of jobs that the experiment ran.

Describe alternatives you've considered A) Get /experiments/{id} returns a full list of all jobs, regardless of whether they were launched in the same iteration or not. This likely makes sense while we have not introduced an external tool to track experiments. B) We adopt MLFlow (and others) abstraction of runs as an iteration of an experiment, meaning that get would return... a list of related runs? [Suggestion, this could be done once we outsource experiment tracking.]