search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.33k stars 505 forks source link

what's the purpose of cl_init #215

Closed lihuiliullh closed 1 year ago

lihuiliullh commented 1 year ago

May I know what this part of code does? Is the pooler in cl_init the random dropout in your paper?

image

github-actions[bot] commented 1 year ago

Stale issue message

gaotianyu1350 commented 1 year ago

Hi, this is just an init function for the contrastive learning class. The dropout is the standard dropout within the Transformer. Pooler here is just the last layer MLP or average pooler (depending on what type of pooler you choose).