search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.31k stars 502 forks source link

关于 Supervised SimCSE 的 GPU Memory Usage #277

Closed AnonymXXXXX closed 4 months ago

AnonymXXXXX commented 4 months ago

Hi~

根据论文附录，有监督场景下（使用 NLI 数据集）SimCSE-BERT-base 和 SimCSE-RoBERTa-base 的 batch size 为 512，seq len 为 32。请问这样的训练配置需要多少显存（GPU Memory Usage）？以及 SimCSE-BERT-base 和 SimCSE-RoBERTa-base 的显存占用是否一致？（需要据此确定自己的硬件环境，谢谢！）

gaotianyu1350 commented 4 months ago

Hi,

The unsupervised model was trained with 1 3090 and the supervised model was trained with 4x3090.

AnonymXXXXX commented 4 months ago

Ok, that should be 24G*4, thanks for your reply.