search

AIPHES / emnlp19-moverscore

MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance
MIT License
192 stars 31 forks source link

Why the system summaries with only one sentence are ignored in run_summarization.py #12

Open Chen-Wang-CUHK opened 3 years ago

Chen-Wang-CUHK commented 3 years ago

Hi Wei, As indicates in line 81 in run_summarization.py: "if len(annot['text']) > 1:". The system summaries with one sentence are ignored when computing correlation. Totally 246 system summaries are ignored. I am curious about the following questions:

 1. Why do we need to skip them? I think it is more reasonable to include them. 
 2. Do you actually want to use ">=" instead of ">"?
 3. Was this operation applied to the baselines?

Thank you!

Wang