search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.31k stars 502 forks source link

drpout #279

Closed riyajatar37003 closed 2 months ago

riyajatar37003 commented 2 months ago

Hi,

where exactly dropout is being applied ? can anyone point to code/file.

thanks

yaoxingcheng commented 2 months ago

Hi,

We reuse the dropouts implemented by Huggingface's transformers, which are applied to attention blocks and hidden states of each transformer layer. See modeling_bert.py and modeling_roberta.py from transformers' source code for details.

riyajatar37003 commented 2 months ago

Thanks got it.