neulab / ExplainaBoard

Interpretable Evaluation for AI Systems
MIT License
361 stars 36 forks source link

Several things are not documented in text classification tutorial #67

Open neubig opened 2 years ago

neubig commented 2 years ago

I think it would be nice to have a tutorial that is basically the first thing that people look at when they want to figure out how to use ExplainaBoard on their own systems. The text classification tutorial is one good candidate for this, as text classification is a very easy-to-understand task with wide applicability.

However, while it tells you how to run the program and get an output json file, it is not quite useful yet because we need:

My ideal solution would be to link it together with something like what will be implemented in this PR (https://github.com/neulab/ExplainaBoard/issues/60) which will be much more interpretable than a static JSON file, but at least an explanation of the JSON file format and what we could glean from it would be useful.

In addition, some basic functionality of ExplainaBoard is not documented, e.g.:

It'd be nice to work on this documentation!

pfliu-nlp commented 2 years ago

"My ideal solution would be to link it together with something like what will be implemented in this PR"

Yeah, I have similar solution before I read the above sentence. We need some env where cli and visual platform can interact in a real-time fashion. I have mentioned this example.