Test sets for acm_Arab, acq_Arab, ars_Arab too similar to arb_Arab and each other - need flagging

openlanguagedata / flores

The FLORES+ Machine Translation Benchmark

Creative Commons Attribution Share Alike 4.0 International

84 stars 14 forks source link

Test sets for acm_Arab, acq_Arab, ars_Arab too similar to arb_Arab and each other - need flagging #8

Open laurieburchell opened 5 months ago

laurieburchell commented 5 months ago

The FLORES-200 test sets for {acm,acq,ars}_Arab are near identical with the main differences being data localisation. Could a native speaker check these (and possibly retranslate)? They should at least be flagged as potentially unreliable.