THUNLP-MT / THUMT

An open-source neural machine translation toolkit developed by Tsinghua Natural Language Processing Group
BSD 3-Clause "New" or "Revised" License
703 stars 197 forks source link

BLEU scores #32

Closed Julisa-test closed 6 years ago

Julisa-test commented 6 years ago

Hi,the training has been finished, but when I run the follow command, there is an error. Would you mind giving me some guide? Thanks!

(venv-2.7.14) ubuntu@ubuntu:~/python2.7/tensorflow$ multi-bleu.perl -lc newstest2015.tc.en < newstest2015.trans.norm > evalResult multi-bleu.perl: command not found (venv-2.7.14) ubuntu@ubuntu:~/python2.7/tensorflow$ ls THUMT corpus.tc.en newstest2014-deen-ref.en.sgm newstest2015-deen-ref.en.sgm newstest2015.trans newstest2016.tc.en train vocab.de bpe32k dev.tgz newstest2014-deen-src.de.sgm newstest2015-deen-src.de.sgm newstest2015.trans.norm nohup.out train_2 vocab.en corpus.tc.32k.de evalResult newstest2014-deen-src.en.sgm newstest2015-ende-ref.de.sgm newstest2016-deen-ref.en.sgm requirment.txt train_bak corpus.tc.32k.de.shuf info.txt newstest2014.tc.32k.de newstest2015-ende-src.en.sgm newstest2016-deen-src.de.sgm run.txt train_seq2seq corpus.tc.32k.en mteval-14.pl newstest2014.tc.32k.en newstest2015.tc.32k.de newstest2016-ende-ref.de.sgm subword-nmt true.tgz corpus.tc.32k.en.shuf multi-bleu.perl newstest2014.tc.de newstest2015.tc.de newstest2016-ende-src.en.sgm tarin_1 vocab.32k.de.txt corpus.tc.de newstest2014-deen-ref.de.sgm newstest2014.tc.en newstest2015.tc.en newstest2016.tc.de test.py vocab.32k.en.txt When I tried like this, another error is coming (venv-2.7.14) ubuntu@ubuntu:~/python2.7/tensorflow$ perl multi-bleu.perl -lc newstest2015.tc.en < newstest2015.trans.norm > evalResult Use of uninitialized value in division (/) at multi-bleu.perl line 139, <STDIN> line 2169. Use of uninitialized value in division (/) at multi-bleu.perl line 139, <STDIN> line 2169. It is in-advisable to publish scores from multi-bleu.perl. The scores depend on your tokenizer, which is unlikely to be reproducible from your paper or consistent across research groups. Instead you should detokenize then use mteval-v14.pl, which has a standard tokenization. Scores from multi-bleu.perl can still be used for internal purposes when you have a consistent tokenizer.