moses-smt / mosesdecoder

Moses, the machine translation system
http://www.statmt.org/moses
GNU Lesser General Public License v2.1
1.58k stars 778 forks source link

[BLEU] Fix multi-bleu.perl bug (no newline at end of file) #201

Closed louismartin closed 6 years ago

louismartin commented 6 years ago

When reading hypothesis and reference files, multi-bleu.perl uses the chop function to remove the trailing newline character. If one of these files happens to not end with a newline, then chop will remove the last character of the last line (instead of the newline). This causes the BLEU score to be slightly off from its theoretical value. Using the safest chomp function solves this problem, i.e. it only removes newlines when present.

hieuhoang commented 6 years ago

looks good, thanks! Will pull unless anyone has a good reason why I shouldn't

hieuhoang commented 6 years ago

done, thanks