microsoft / DeBERTa

The implementation of DeBERTa
MIT License
1.97k stars 224 forks source link

where is the absolute position embeddings? #52

Open ylwangy opened 3 years ago

ylwangy commented 3 years ago

The paper says you add the absolte position embeddings after all Transformer layers, before softmax layer for MLM, however, I could not find these parameters.

looking forward to your response. Thank you