search

zeno-ml / zeno-build

Build, evaluate, understand, and fix LLM-based apps
MIT License
483 stars 33 forks source link

Visualization: Text Classification #10

Closed neubig closed 1 year ago

neubig commented 1 year ago

Text classification outputs are not yet visualized using Zeno.

neubig commented 1 year ago

Here are results from a run of text classification training.

text_classification_results.zip

The place where visualization needs to be implemented is here: https://github.com/zeno-ml/llm-compare/blob/main/tasks/text_classification/main.py#L67

If you read in all_runs.json from the attached file it should give you the contents of serialized_result.

cabreraalex commented 1 year ago

Looks like there's not an function to get the raw data from the modeling.py file.

More generally, I wonder if the package should provide more structured interfaces for each task, for example the functions that must be implemented.

neubig commented 1 year ago

Looks like there's not an function to get the raw data from the modeling.py file.

You mean the dataset? Yeah, that's probably correct. We should probably modify the get_references() file to return information about the dataset as a whole.

More generally, I wonder if the package should provide more structured interfaces for each task, for example the functions that must be implemented.

I agree. What I was thinking is that we start by copy-pasting one or two times, then try to standardize the interface a bit. I tried to write it in a somewhat general terms in the first place so hopefully it won't be too hard to generalize later.

cabreraalex commented 1 year ago

Got it, that makes tons of sense - I'll update the classification example to make a bit more sense with the final analysis part.