search

graykode / nlp-tutorial

Natural Language Processing Tutorial for Deep Learning Researchers
https://www.reddit.com/r/MachineLearning/comments/amfinl/project_nlptutoral_repository_who_is_studying/
MIT License
14.07k stars 3.91k forks source link

Why is first parameter `src_vocab_size`? #30

Closed yangdechuan closed 5 years ago

yangdechuan commented 5 years ago

https://github.com/graykode/nlp-tutorial/blob/3b3a80dc63e69935731bcf09c951eb371692af8f/5-1.Transformer/Transformer(Greedy_decoder)-Torch.py#L141

position encoding table should be (src_len, d_model). Why (src_vocab_size, d_model) here?

graykode commented 5 years ago

Oops.. Thanks for right catching In Encoder src_len+1 is right, and as same tgt_len+1 in Decoder. I will fixed as soon as possible Thanks