search

eaclark07 / sms

43 stars 11 forks source link

About the Summaries Dataset #4

Closed Chen-Wang-CUHK closed 3 years ago

Chen-Wang-CUHK commented 3 years ago

Hi Elizabeth, Thank you for your interesting work. Is it possible to share the summaries dataset that you used to get the results of Table 2 in the paper? I noted that in the raw data from Chaganty et al. (2018), there are some instances only containing one or two summary systems' human scores instead of the complete five summarization systems. How do you handle these cases? remove them or keep them? Thank you!

eaclark07 commented 3 years ago

We only included the instances that have scores from 2 or more human judges. So you can remove any instances with fewer than 2 scores.

Chen-Wang-CUHK commented 3 years ago

We only included the instances that have scores from 2 or more human judges. So you can remove any instances with fewer than 2 scores.

Dear Elizabeth, thank you for your clarification.