shape match for mul？ - Githubissues

codertimo / BERT-pytorch

Google AI 2018 BERT pytorch implementation

Apache License 2.0

6.11k stars 1.29k forks source link

shape match for mul？ #37

Closed guotong1988 closed 5 years ago

guotong1988 commented 5 years ago

https://github.com/codertimo/BERT-pytorch/blob/alpha0.0.1a5/bert_pytorch/model/embedding/position.py#L18

What are the shapes of position and div_term?

nateraw commented 5 years ago

It's fairly easy to step through the code to find the shape. d_model is the embedding size and max_len is the maximum length of a sequence. Hope this helps