edit segment indices and embedding numbers for padding

dhlee347 / pytorchic-bert

Pytorch Implementation of Google BERT

Apache License 2.0

589 stars 181 forks source link

edit segment indices and embedding numbers for padding #11

Closed AppleHolic closed 5 years ago

AppleHolic commented 5 years ago

As I said https://github.com/dhlee347/pytorchic-bert/issues/10, there is a bug in segment index. Fix that.

vincentlux commented 5 years ago

If self.seg_embed = nn.Embedding(config.n_segments + 1, config.dim), it is not possible to load tf checkpoint file since the original model has dimension (2, 768).

AssertionError: Dim Mismatch: (3, 768) vs (2, 768) ; bert/embeddings/token_type_embeddings