search

howardyclo / pytorch-seq2seq-example

Fully batched seq2seq example based on practical-pytorch, and more extra features.
76 stars 17 forks source link

replace for-loop in forward decoder #2

Closed Tangzy7 closed 5 years ago

Tangzy7 commented 6 years ago

Hello I also found that forward decoder (for-loop) is time-consuming, do you have a good solution instead of for-loop?

howardyclo commented 6 years ago

@Tangzy7 Hello, I study the codebase of OpenNMT-py, and found that if the decoder without input feed can be implemented without for loop. Please refer to: https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/Models.py#L388

But if using the decoder with input feed, I think it must be implemented with for-loop: https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/Models.py#L490

In inference mode, a for-loop is required since we don't know the ground-truth targets (model predict token by its previous token).