the total number of trainable parameters in 12 layer BERT

dhlee347 / pytorchic-bert

Pytorch Implementation of Google BERT

Apache License 2.0

589 stars 181 forks source link

the total number of trainable parameters in 12 layer BERT #20

Closed mingbocui closed 4 years ago

mingbocui commented 4 years ago

Hi,

Have you checked the total number of parameters in the model? I checked that it is 220 million, which is more than the 110 million parameters presented in the original BERT-base model. Hope for your reply! Thanks!

mingbocui commented 4 years ago

sorry for my bad, I calculated it wrongly :(