search

GEM-benchmark / GEM-metrics

Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60 stars 20 forks source link

Adding Translation error rate, TER metric #82

Closed leoribeiro closed 2 years ago

leoribeiro commented 2 years ago

This PR adds the metric Translation error rate (TER), using the sacrebleu implementation. TER is an error metric for machine translation that measures the number of edits required to change a system output into one of the references.

tuetschek commented 2 years ago

The code looks good to me, thanks @leoribeiro ! Could you please also add a test for TER? See https://github.com/GEM-benchmark/GEM-metrics/tree/main/tests for instructions.

leoribeiro commented 2 years ago

Test added.

sebastianGehrmann commented 2 years ago

Kicked off the test. If it passes, we can merge, @tuetschek . Thank you @leoribeiro!

tuetschek commented 2 years ago

All looks good. Thanks @leoribeiro @sebastianGehrmann !