search

hengruo / QANet-pytorch

A PyTorch implementation of QANet.
MIT License
345 stars 67 forks source link

Position Encoding #3

Closed InitialBug closed 6 years ago

InitialBug commented 6 years ago

In the original paper 《attention is all you need》, it seems the position encoding is direct computed rather than trained. But in your code, the final parameter is wrapped with torch.nn.Parameter, is that OK?

hengruo commented 6 years ago

@InitialBug You are right. I should set requires_grad=False. Thanks!