capitalone / rubicon-ml

Capture all information throughout your model's development in a reproducible way and tie results directly to the model code!
https://capitalone.github.io/rubicon-ml/
Apache License 2.0
129 stars 34 forks source link

create intake source for metric list comparisons #455

Closed ryanSoley closed 3 months ago

ryanSoley commented 4 months ago

Is your enhancement request related to a problem? Please describe

we would like to be able to share visualizations alongside the projects and experiments in the intake catalogs. for example, catalog.read() returns projects and experiments. we'd like to ultimately have something like catalog.serve() that mimics the viz module's serve functionality to launch a dashboard with the experiments returned by catalog.read()

in #441 we created a base source for visualizations. in #442 we created a source for the experiments table. we will need a concrete implementation for each visualization like the one implemented for #442. this issue will track the creation of a source for the metric lists comparisons

Describe the solution you'd like

add a new MetricListComparisonDataSource that extends VizDataSourceMixin. the __init__ function should take in any additional arguments specific to the MetricListsComparison. _get_schema should create a visualization and set it as the _visualization attribute, as well as update the metadata with any required attributes

Additional context

at this point, the visualization stored in _visualization does not need to have any experiments in it - we'll get to that later

this should be implemented very similarly to the solutions to #442, #453 and #454. it may even be possible to move some common functionality between the existing sources and the new MetricListComparisonDataSource to the base VizDataSourceMixin if its shared across both viz's implementations

ryanSoley commented 3 months ago

closed via #467