search

facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
MIT License
30.18k stars 6.37k forks source link

about generate.py #409

Closed zll0000 closed 5 years ago

zll0000 commented 5 years ago

after I trained an en-de translation model,I use the generate.py to translate source sequence, "python generate.py data-bin/wmt14_en_de/ --path /checkpoints/transformer/checkpoint_best.pt --beam 5 --remove-bpe --source-lang en --target-lang de --score-reference" however I got the translation result:

S-29 The theatre producer has thus combined elements from the autobiographically inspired novel " Der Leibeigene " ( 1987 ) [ The Bondsman ] featuring prose miniatures from " Leichnam , seine Familie belauernd " ( 2003 ) [ Corpse , stalking his family ] . T-29 Der Theatermacher verbindet etwa Elemente aus dem autobiografisch geprägten Roman " Der Leibeigene " ( 1987 ) mit Prosaminiaturen aus " Leichnam , seine Familie belauernd " ( 2003 ) . H-29 -8.418146133422852 Der Theatermacher verbindet etwa Elemente aus dem autobiografisch geprägten Roman " Der Leibeigene " ( 1987 ) mit Prosaminiaturen aus " Leichnam , seine Familie belauernd " ( 2003 ) . P-29 -7.0659 -11.4109 -11.4834 -10.7223 -8.6708 -10.4547 -6.4828 -5.4867 -11.5541 -9.2510 -11.6334 -9.0156 -11.7988 -11.5758 -5.6855 -7.0659 -9.6949 -9.3726 -10.3977 -5.6855 -6.1851 -11.5358 -6.4151 -5.3088 -9.7986 -10.0067 -10.3855 -11.2690 -6.4828 -5.6855 -9.6949 -10.9483 -7.0070 -3.1608 -7.7121 -9.5281 -9.6277 -9.0214 -9.7405 -8.6108 -5.6855 -6.1851 -9.0303 -6.4151 -3.6632 -3.6224

and the blue " Generate test with beam=5: BLEU4 = 97.33, 98.8/97.8/96.8/95.9 (BP=1.000, ratio=1.000, syslen=64506, reflen=64506)"

I have some questions:

  1. what does the "P-" and "H-" lines means? is translation result?or something else 2.why the "T-" line are test reference not the translation result and the "T-line " is the same with "H-" line 3.if I want to translate an source text file ,how to do it? and get the result
huihuifan commented 5 years ago

First, score-reference scores the probability of the reference, which is why H (hypothesis) and T (target) is the same. P is the per token likelihood. There's one more than H here, because P scores eos as well.

To answer your question 3- you can either use interactive.py or you can binarize your data with the same dictionary as the trained model. However, for both, you need to not use the --score-reference option.