search

autoliuweijie / FastBERT

The score code of FastBERT (ACL2020)
https://www.aclweb.org/anthology/2020.acl-main.537/
604 stars 90 forks source link

I'm curious why you set segment_embedding's first dimension to 3. #36

Open wonbeeny opened 3 years ago

wonbeeny commented 3 years ago

First of all, thanks for your kind offer.

Why did you set segment_embedding's first dimension to 3 ??

This is on path [FastBERT/uer/layers/embeddings.py] (line 18)

Is this part flexible depending on the model architecture? The paper does not have this content, so I asked a question.

Domifance commented 3 years ago

You look at the code, It has nothing to do with the structure of the model. It's just a convenient to model implementation, in fact,segment_embedding first dimension in pre_trained model is all zeros.