zeno-ml / zeno

AI Data Management & Evaluation Platform
https://zenoml.com
MIT License
213 stars 10 forks source link

How to find and display only some models by traits #798

Open neubig opened 1 year ago

neubig commented 1 year ago

I tried to run Zeno with 100 models, and then only wanted to take a look at the top-performing ones. Is there a good way to do this?

Right now I am stuck in reports view, and see this with a bar chart:

Screen Shot 2023-05-25 at 12 19 51 AM

this with a table:

Screen Shot 2023-05-25 at 12 23 32 AM

and this with a line chart:

Screen Shot 2023-05-25 at 12 22 55 AM

The line chart does allow me to find at least one top-scoring models by mousing-over the highest points (and some of the dots are hidden).

It'd be nice if it were possible to:

  1. Sort the table by score (also maybe swap the rows and columns?)
  2. Even better, "slice" models by accuracy or other features, to only view a certain portion of them. It would also be useful to be able to do this with other traits, such as the model hyper-parameters.
cabreraalex commented 1 year ago

🤯 I love it super power user. Both of those sound reasonable, with the table being the easiest. We'll have to think of how to filter out models though...

cabreraalex commented 1 year ago

@stevenyh3 could you look into potentially supporting the table where the rows are models and column are metrics/maybe some simple sorting? This might already be possible.

stevenyh3 commented 1 year ago

For now, I think the rows and columns are able to swap like the attached picture. But metrics are only feasible in the layer dropdown now. I'll try to support multiple metrics as columns(x-axis), and add sort feature in the near future.

image