Hello author, I am working on an unsupervised neural machine translation task of zh-en. First, I used 4 GPUs for pre-training. The results are as follows. Then I performed unsupervised translation training. The BLEU score is very low and the degree of confusion is increasing.
pre-training：
INFO - 12/26/20 06:58:39 - 3:31:28 - valid_en_mlm_ppl -> 13.285079
INFO - 12/26/20 06:58:39 - 3:31:28 - valid_en_mlm_acc -> 51.750151
INFO - 12/26/20 06:58:39 - 3:31:28 - valid_zh_mlm_ppl -> 12.769013
INFO - 12/26/20 06:58:39 - 3:31:28 - valid_zh_mlm_acc -> 50.882223
INFO - 12/26/20 06:58:39 - 3:31:28 - valid_mlm_ppl -> 13.027046
INFO - 12/26/20 06:58:39 - 3:31:28 - valid_mlm_acc -> 51.316187
INFO - 12/26/20 06:58:39 - 3:31:28 - test_en_mlm_ppl -> 15.987829
INFO - 12/26/20 06:58:39 - 3:31:28 - test_en_mlm_acc -> 50.794515
INFO - 12/26/20 06:58:39 - 3:31:28 - test_zh_mlm_ppl -> 11.470636
INFO - 12/26/20 06:58:39 - 3:31:28 - test_zh_mlm_acc -> 52.364314
INFO - 12/26/20 06:58:39 - 3:31:28 - test_mlm_ppl -> 13.729233
INFO - 12/26/20 06:58:39 - 3:31:28 - test_mlm_acc -> 51.579414
training：
INFO - 12/21/20 03:39:10 - 2:38:11 - ============ End of epoch 0 ============
INFO - 12/21/20 03:39:52 - 2:38:53 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp0.en-zh.valid.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.en-zh.valid.txt : 5.490000
INFO - 12/21/20 03:40:29 - 2:39:29 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp0.zh-en.valid.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.zh-en.valid.txt : 5.050000
INFO - 12/21/20 03:41:09 - 2:40:10 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp0.en-zh.test.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.en-zh.test.txt : 6.210000
INFO - 12/21/20 03:41:44 - 2:40:44 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp0.zh-en.test.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.zh-en.test.txt : 5.280000
INFO - 12/21/20 03:41:44 - 2:40:44 - epoch -> 0.000000
INFO - 12/21/20 03:41:44 - 2:40:44 - valid_en-zh_mt_ppl -> 82.061756
INFO - 12/21/20 03:41:44 - 2:40:44 - valid_en-zh_mt_acc -> 33.095098
INFO - 12/21/20 03:41:44 - 2:40:44 - valid_en-zh_mt_bleu -> 5.490000
INFO - 12/21/20 03:41:44 - 2:40:44 - valid_zh-en_mt_ppl -> 83.254574
INFO - 12/21/20 03:41:44 - 2:40:44 - valid_zh-en_mt_acc -> 33.050408
INFO - 12/21/20 03:41:44 - 2:40:44 - valid_zh-en_mt_bleu -> 5.050000
INFO - 12/21/20 03:41:44 - 2:40:44 - test_en-zh_mt_ppl -> 72.410014
INFO - 12/21/20 03:41:44 - 2:40:44 - test_en-zh_mt_acc -> 35.406409
INFO - 12/21/20 03:41:44 - 2:40:44 - test_en-zh_mt_bleu -> 6.210000
INFO - 12/21/20 03:41:44 - 2:40:44 - test_zh-en_mt_ppl -> 77.783462
INFO - 12/21/20 03:41:44 - 2:40:44 - test_zh-en_mt_acc -> 34.309790
INFO - 12/21/20 03:41:44 - 2:40:44 - test_zh-en_mt_bleu -> 5.280000
INFO - 12/21/20 11:30:37 - 10:29:37 - ============ End of epoch 3 ============
INFO - 12/21/20 11:31:17 - 10:30:18 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp3.en-zh.valid.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.en-zh.valid.txt : 5.320000
INFO - 12/21/20 11:31:56 - 10:30:57 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp3.zh-en.valid.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.zh-en.valid.txt : 5.110000
INFO - 12/21/20 11:32:34 - 10:31:34 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp3.en-zh.test.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.en-zh.test.txt : 6.110000
INFO - 12/21/20 11:33:10 - 10:32:11 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp3.zh-en.test.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.zh-en.test.txt : 5.770000
INFO - 12/21/20 11:33:10 - 10:32:11 - epoch -> 3.000000
INFO - 12/21/20 11:33:10 - 10:32:11 - valid_en-zh_mt_ppl -> 225.084927
INFO - 12/21/20 11:33:10 - 10:32:11 - valid_en-zh_mt_acc -> 30.757588
INFO - 12/21/20 11:33:10 - 10:32:11 - valid_en-zh_mt_bleu -> 5.320000
INFO - 12/21/20 11:33:10 - 10:32:11 - valid_zh-en_mt_ppl -> 211.707106
INFO - 12/21/20 11:33:10 - 10:32:11 - valid_zh-en_mt_acc -> 29.768266
INFO - 12/21/20 11:33:10 - 10:32:11 - valid_zh-en_mt_bleu -> 5.110000
INFO - 12/21/20 11:33:10 - 10:32:11 - test_en-zh_mt_ppl -> 188.079560
INFO - 12/21/20 11:33:10 - 10:32:11 - test_en-zh_mt_acc -> 32.999769
INFO - 12/21/20 11:33:10 - 10:32:11 - test_en-zh_mt_bleu -> 6.110000
INFO - 12/21/20 11:33:10 - 10:32:11 - test_zh-en_mt_ppl -> 192.445832
INFO - 12/21/20 11:33:10 - 10:32:11 - test_zh-en_mt_acc -> 31.239425
INFO - 12/21/20 11:33:10 - 10:32:11 - test_zh-en_mt_bleu -> 5.770000
Hello author, I am working on an unsupervised neural machine translation task of zh-en. First, I used 4 GPUs for pre-training. The results are as follows. Then I performed unsupervised translation training. The BLEU score is very low and the degree of confusion is increasing. pre-training： INFO - 12/26/20 06:58:39 - 3:31:28 - valid_en_mlm_ppl -> 13.285079 INFO - 12/26/20 06:58:39 - 3:31:28 - valid_en_mlm_acc -> 51.750151 INFO - 12/26/20 06:58:39 - 3:31:28 - valid_zh_mlm_ppl -> 12.769013 INFO - 12/26/20 06:58:39 - 3:31:28 - valid_zh_mlm_acc -> 50.882223 INFO - 12/26/20 06:58:39 - 3:31:28 - valid_mlm_ppl -> 13.027046 INFO - 12/26/20 06:58:39 - 3:31:28 - valid_mlm_acc -> 51.316187 INFO - 12/26/20 06:58:39 - 3:31:28 - test_en_mlm_ppl -> 15.987829 INFO - 12/26/20 06:58:39 - 3:31:28 - test_en_mlm_acc -> 50.794515 INFO - 12/26/20 06:58:39 - 3:31:28 - test_zh_mlm_ppl -> 11.470636 INFO - 12/26/20 06:58:39 - 3:31:28 - test_zh_mlm_acc -> 52.364314 INFO - 12/26/20 06:58:39 - 3:31:28 - test_mlm_ppl -> 13.729233 INFO - 12/26/20 06:58:39 - 3:31:28 - test_mlm_acc -> 51.579414
training： INFO - 12/21/20 03:39:10 - 2:38:11 - ============ End of epoch 0 ============ INFO - 12/21/20 03:39:52 - 2:38:53 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp0.en-zh.valid.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.en-zh.valid.txt : 5.490000 INFO - 12/21/20 03:40:29 - 2:39:29 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp0.zh-en.valid.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.zh-en.valid.txt : 5.050000 INFO - 12/21/20 03:41:09 - 2:40:10 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp0.en-zh.test.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.en-zh.test.txt : 6.210000 INFO - 12/21/20 03:41:44 - 2:40:44 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp0.zh-en.test.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.zh-en.test.txt : 5.280000 INFO - 12/21/20 03:41:44 - 2:40:44 - epoch -> 0.000000 INFO - 12/21/20 03:41:44 - 2:40:44 - valid_en-zh_mt_ppl -> 82.061756 INFO - 12/21/20 03:41:44 - 2:40:44 - valid_en-zh_mt_acc -> 33.095098 INFO - 12/21/20 03:41:44 - 2:40:44 - valid_en-zh_mt_bleu -> 5.490000 INFO - 12/21/20 03:41:44 - 2:40:44 - valid_zh-en_mt_ppl -> 83.254574 INFO - 12/21/20 03:41:44 - 2:40:44 - valid_zh-en_mt_acc -> 33.050408 INFO - 12/21/20 03:41:44 - 2:40:44 - valid_zh-en_mt_bleu -> 5.050000 INFO - 12/21/20 03:41:44 - 2:40:44 - test_en-zh_mt_ppl -> 72.410014 INFO - 12/21/20 03:41:44 - 2:40:44 - test_en-zh_mt_acc -> 35.406409 INFO - 12/21/20 03:41:44 - 2:40:44 - test_en-zh_mt_bleu -> 6.210000 INFO - 12/21/20 03:41:44 - 2:40:44 - test_zh-en_mt_ppl -> 77.783462 INFO - 12/21/20 03:41:44 - 2:40:44 - test_zh-en_mt_acc -> 34.309790 INFO - 12/21/20 03:41:44 - 2:40:44 - test_zh-en_mt_bleu -> 5.280000 INFO - 12/21/20 11:30:37 - 10:29:37 - ============ End of epoch 3 ============ INFO - 12/21/20 11:31:17 - 10:30:18 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp3.en-zh.valid.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.en-zh.valid.txt : 5.320000 INFO - 12/21/20 11:31:56 - 10:30:57 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp3.zh-en.valid.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.zh-en.valid.txt : 5.110000 INFO - 12/21/20 11:32:34 - 10:31:34 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp3.en-zh.test.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.en-zh.test.txt : 6.110000 INFO - 12/21/20 11:33:10 - 10:32:11 - BLEU ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/hyp3.zh-en.test.txt ./dumped/unsupMT_zhen/vlt9yq3mje/hypotheses/ref.zh-en.test.txt : 5.770000 INFO - 12/21/20 11:33:10 - 10:32:11 - epoch -> 3.000000 INFO - 12/21/20 11:33:10 - 10:32:11 - valid_en-zh_mt_ppl -> 225.084927 INFO - 12/21/20 11:33:10 - 10:32:11 - valid_en-zh_mt_acc -> 30.757588 INFO - 12/21/20 11:33:10 - 10:32:11 - valid_en-zh_mt_bleu -> 5.320000 INFO - 12/21/20 11:33:10 - 10:32:11 - valid_zh-en_mt_ppl -> 211.707106 INFO - 12/21/20 11:33:10 - 10:32:11 - valid_zh-en_mt_acc -> 29.768266 INFO - 12/21/20 11:33:10 - 10:32:11 - valid_zh-en_mt_bleu -> 5.110000 INFO - 12/21/20 11:33:10 - 10:32:11 - test_en-zh_mt_ppl -> 188.079560 INFO - 12/21/20 11:33:10 - 10:32:11 - test_en-zh_mt_acc -> 32.999769 INFO - 12/21/20 11:33:10 - 10:32:11 - test_en-zh_mt_bleu -> 6.110000 INFO - 12/21/20 11:33:10 - 10:32:11 - test_zh-en_mt_ppl -> 192.445832 INFO - 12/21/20 11:33:10 - 10:32:11 - test_zh-en_mt_acc -> 31.239425 INFO - 12/21/20 11:33:10 - 10:32:11 - test_zh-en_mt_bleu -> 5.770000