facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
MIT License
30.57k stars 6.41k forks source link

How to generate hypothese with fconv for DrNMT? (ende) #5563

Open 123THU opened 2 weeks ago

123THU commented 2 weeks ago

❓ Questions and Help

I followed the instructions from DrNMT to train a base MT model(fconv) first, and then hoped to use the MT model to prepare three files: source, target and hypothese. However, when I started to use the generated hypotheses file, it showed:

_AssertionError: /mnt/workspace/lastversion/venv/fairseq/examples/discriminative_reranking_nmt/data/trainhypothese.txt should contain 11883537 hypotheses but only has 3961176. (--beam=3)

So I want to ask what are the correct files(source and target, I choose train.en, train.de)), and how should I generate the correct hypothese file.

Thank you so much for relpying!!!!

This is the code I use to generate hypotheses with fconv

fairseq-generate data-bin/wmt17_en_de \ --path checkpoints/fconv_wmt_en_de/checkpoint_best.pt \ --beam 3 --remove-bpe --gen-subset train > output_train.txt

environment