search

lil-lab / newsroom

Tools for downloading and analyzing summaries and evaluating summarization systems. https://summari.es/
Other
147 stars 24 forks source link

Dataset Diversity Analysis #25

Closed StevenLau6 closed 2 years ago

StevenLau6 commented 2 years ago

Dear Artzi @yoavartzi, first of all, I want to thank you for your great work.
I have two questions about the diversity analysis in your paper. I used your code to calculate the coverage, density, and compression. And then, I used the seaborn.kdeplot to visualize the results. But I found my result was different from figure 4 in your paper. The coverage score seems much lower. My questions are: 1) Did you divide the coverage scores with the maximum value or conduct the min-max normalization? 2) Did you randomly sample or use the entire dataset (training set) to calculate these three metrics? Thank you again for your help.

My result is here: image

yoavartzi commented 2 years ago

Unfortunately, I don’t have information beyond what’s in the paper and in the released code at this point. Sorry