Added greedy decoder input

graykode / nlp-tutorial

Natural Language Processing Tutorial for Deep Learning Researchers

https://www.reddit.com/r/MachineLearning/comments/amfinl/project_nlptutoral_repository_who_is_studying/

MIT License

14.07k stars 3.91k forks source link

Added greedy decoder input #6

Closed dmmiller612 closed 5 years ago

dmmiller612 commented 5 years ago

In this PR, I added a greedy decoder function that generates the decoder input for inference. This is important for translating sentences as we don't know the target input beforehand. In the paper, they mentioned that they ran Beam Search with a k=4. In the greedy approach, k = 1.

dmmiller612 commented 5 years ago

Weird, I can see that a message was added to this thread from my email, but I can't see it here. Maybe Github is acting up?

graykode commented 5 years ago

Hello, dmmiller612!! Thanks for your pull request! Your code is very nice for implemented Transformer. I'll add self-attention Mask on Decoder in Transformer Code However, My main point is show to beginner more easily! So, Could you send me new Pull Request about new file name like Transformer(Greedy_decoder)-Torch.py ? My mean is your Pr is very good, but not overlapping this code, apply Pr in new Code Transformer(Greedy_decoder)-Torch.py @dmmiller612 I will wait for your new Pull Request Thanks you :D

dmmiller612 commented 5 years ago

Sounds good. I'll try to get it done tonight.

graykode commented 5 years ago

@dmmiller612 Good. I'll also add your code to ipynb file in Google Colab for beginner