GEM-benchmark / GEM-metrics

Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60 stars 20 forks source link

MATTR metric #71

Open ndaheim opened 2 years ago

ndaheim commented 2 years ago

https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.248.5206&rep=rep1&type=pdf

While MSTTR calculates TTR for non-overlapping segments, MATTR uses a sliding window.

Easy to implement and doesn't suffer from segments being disregarded as is the case in MSTTR (for example, text length 19, window size 10 means that only the first 10 tokens are considered).