Open longdct opened 3 years ago
This issue has been automatically marked as stale. If this issue is still affecting you, please leave any comment (for example, "bump"), and we'll keep it open. We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!
Reproduce results from mBART paper on IWSLT15 En-Vi dataset
What is your question?
I try to follow the mBART example and finetune on IWSLT15 En-Vi dataset to reproduce the result from the paper. However, after running multiple times with different seed, I only obtained about 34.1 BLEU for En->Vi and 34.6 BLEU for Vi->En, compared with 36.1 and 35.4 from the paper.
Code
Since using
fairseq-generate
from the example with output hypothesis sentence without space (issue #3103), I log the output offairseq-generate
and extract sentences on lineS
,T
andD
to calculate BLEU using sacrebleuWhat's your environment?
pip
, source): pip