search

neulab / explainaboard_web

MIT License
8 stars 2 forks source link

download all analysis charts #497

Closed qjiang002 closed 1 year ago

qjiang002 commented 1 year ago

This PR is related to issue #426 - 10(b)

Export all the charts in the analysis report

Download a zip folder with all the charts in Overall Performance and Fine-grained Analysis. Zip folder name: ExplainaBoardAnalysis{system names}

Screen Shot 2022-11-07 at 4 53 19 PM

Screen Shot 2022-11-07 at 4 51 19 PM

Problems

The current implementation uses an html instance reference to refer to the ReactEChart, which has the following problems:

Do you have any suggestion about this? Thanks!

Accuracy by the average rank of each word based on its frequency in training set

pfliu-nlp commented 1 year ago

Looks good. I agree that it would super useful if these charts could be downloaded in a tsv, csv or JSON format.

qjiang002 commented 1 year ago

I think @noelchen90 is working on displaying the latex format of each chart using a modal next to each chart. Can you please explain what kind of tsv/csv/json format of chart is expected? Is it similar to the json report produced by SDK?

pfliu-nlp commented 1 year ago

Hi, @qjiang002 I guess we have the following things:

  1. Fine-grained analysis
    • [x] (1). bar chart is downloadable as a png image
    • [ ] (2). bar chart is downloadable as latex code
    • [ ] (3). bar chart is downloadable as a JSON file (describe the bucket information, such as bucket size, performance, confidence interval)

If @noelchen90 is working on (2), maybe (3) is not urgent so far.

  1. Detailed example analysis

    • [x] the table is downloadable as a tsv file

  2. holistically

    • [ ] (1) the whole report JSON file is downloadable
    • [x] (2) all the figures are downloadable

You have finished (2), and I think (1) is not urgent so far or not necessary.

qjiang002 commented 1 year ago

Hi @pfliu-nlp , thanks for the elaboration. This PR aims to download all bar charts as png. If so, a json report like the one generated by SDK may be redundant and less readable. I think it will be better to have a more readable pdf/txt/md report that can describe the analysis in natural language, but it may need more design and can be saved for future development.

pfliu-nlp commented 1 year ago

I think it will be better to have a more readable pdf/txt/md report that can describe the analysis in natural language, but it may need more design and can be saved for future development.

Agree, we are actually implementing some backend support for such a feature, which hopefully could be deployed to the web soon.