kernel size in position-wise ffn

kolloldas / torchnlp

Easy to use NLP library built on PyTorch and TorchText

Apache License 2.0

254 stars 44 forks source link

kernel size in position-wise ffn #17

Open wangjs9 opened 4 years ago

wangjs9 commented 4 years ago

In the paper attention is all you need, the kernel size of convolutions is set to 1. But I find in this implement, this value is 3. Therefore, I am asking the reason.