search

GEM-benchmark / GEM-metrics

Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60 stars 20 forks source link

#3 and necessary changes for new sacrebleu version #67

Closed ndaheim closed 2 years ago

ndaheim commented 2 years ago

Adds chrF (https://aclanthology.org/W15-3049.pdf), chrF+ and chrF++ (https://aclanthology.org/W17-4770.pdf) by wrapping the corresponding function from sacrebleu.

Increases the version requirement of sacrebleu in the requirements.txt and adds the required changes to tokenization in sari.

sebastianGehrmann commented 2 years ago

The test is failing right now, any idea why?

tuetschek commented 2 years ago

The test is failing right now, any idea why?

@sebastianGehrmann I think this is unrelated – it's MSTTR, #55. ChrF seems to be passing.

ndaheim commented 2 years ago

The test is failing right now, any idea why?

@sebastianGehrmann I think this is unrelated – it's MSTTR, #55. ChrF seems to be passing.

Exactly, these are the tests that are currently still failing.