search

microsoft / DeBERTa

The implementation of DeBERTa
MIT License
1.91k stars 216 forks source link

How to get sentence embedding from DeBERTa for SNLI Dataset? #60

Closed akshat-suwalka closed 2 years ago

akshat-suwalka commented 2 years ago

Hi, I want a single vector from hugging face DeBERTA-base model for doing semantic similarity.... Can i use the first vector of DeBERTa to represent the whole sentence OR Can i use the sum of first and last vector to represent sentence embedding??? Can anybody guide me how to use deberta to get the accuracy on snli dataset??

Thanks

BigBird01 commented 2 years ago

You can use either of them based on our evaluation results. Generally, we use the first token, i.e. [CLS].