Fix non-matching tensor with odd embedding size.

codertimo / BERT-pytorch

Google AI 2018 BERT pytorch implementation

Apache License 2.0

6.11k stars 1.29k forks source link

Fix non-matching tensor with odd embedding size. #54

Open zmy opened 5 years ago

zmy commented 5 years ago

If d_model integer parameter is odd rather than even, then tensor size is not matching (by a difference of 1) on the left and right sides of the assignment:

pe[:, 1::2] = torch.cos(position * div_term)

This pull request fixes the problem.