Why does unigrams and bigrams give the same result?

AIPHES / emnlp19-moverscore

MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance

MIT License

197 stars 34 forks source link

Why does unigrams and bigrams give the same result? #19

Open mukhal opened 3 years ago

mukhal commented 3 years ago

Hello,

I am running MOVERScore on summarization outputs, with both n_gram=1 and n_gram=2. Surprisingly, I am getting the exactly same score in both cases. Shouldn't there be a difference, even if small?

andyweizhao commented 3 years ago

@mohammadKhalifa Thanks a lot for your interest! I believe this is because the word_mover_score function is imported from moverscore_v2 instead of moverscore. For simplicity, the features of power-mean and n-gram are disregarded in the moverscore_v2.py to accelerate the speed, while moverscore.py with full features is reserved for the reproducibility.