search

tensorflow / nmt

TensorFlow Neural Machine Translation Tutorial
Apache License 2.0
6.37k stars 1.96k forks source link

beam search broken when time_major=False #180

Closed bastings closed 6 years ago

bastings commented 6 years ago

It looks like beam search fails when time_major is set to False.

I've tested on IWSLT15 vi->en with beam_width 10.

With time_major true, I get >0.0 BLEU scores after 1 epoch. With time_major false, BLEU scores remain 0.0 after many epochs (while perplexity goes down.)

I'm not yet sure what causes this.

Tested on tf-1.2 branch, but probably applies to master as well.

bastings commented 6 years ago

Example broken output on IWSLT Vi-En:

How How How How How How How How How How do do do do do do do do do do I I I I I I I I I I tell tell tell tell tell tell tell tell tell tell ...

oahziur commented 6 years ago

@bastings Thanks, there is a bug in the model.decode when we reshape the final output. I will update with a fix soon.