Add more metrics, esp. for Delay: Average Lagging

ELITR / SLTev

SLTev is a tool for comprehensive evaluation of (simultaneous) spoken language translation.

8 stars 3 forks source link

Open obo opened 2 years ago

obo commented 2 years ago

Average Lagging is one of the popular methods of estimating delay; it is used, i.a. in SimulEval (https://aclanthology.org/2020.emnlp-demos.19/)

Add AL to SLTev in two variants:

the original one (https://aclanthology.org/P19-1289/)
the improvement which avoids negative values; see Section 5.4 of https://aclanthology.org/2022.iwslt-1.24.pdf