search

mjpost / sacrebleu

Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons
Apache License 2.0
1.03k stars 162 forks source link

wmt23 archive uses `translator="refA"` #260

Closed martinpopel closed 5 months ago

martinpopel commented 5 months ago

unlike in previous years, where it used translator="A" See any file in https://github.com/wmt-conference/wmt23-news-systems/tree/master/xml versus e.g. https://github.com/wmt-conference/wmt22-news-systems/tree/master/xml

We have two options:

  1. accept this PR
  2. fix the wmt23-news-systems xml files (release v.0.1)

@kocmitom @mjpost @ozancaglayan what do you prefer?

martinpopel commented 5 months ago

OK, let's merge this PR now (as it fixes #261, i.e. there are users who need the fix). If someone (@kocmitom) publishes a new version (v.0.2) of wmt23-news-systems, it will be easy to change it back from refA to A (and the version from v.0.1 to v.0.2).