finetune on IMDB dataset

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

MIT License

3.36k stars 507 forks source link

finetune on IMDB dataset #165

Closed ShaobinChen-AH closed 2 years ago

ShaobinChen-AH commented 2 years ago

Hello! Glad to see your excellent work! I retrained unsup simcse on my device following readme document and finetuned it on imdb dataset to improve the performance of sentiment classification. I also converted simcse to huggingface format after training the simcse. But when I load simcse when finetuning on imdb, I got the following error: I have no idea about this. Could you point out what mistakes I made. Thanks a lot.

gaotianyu1350 commented 2 years ago

It seems that you are using a different type of layer norm (layer norm's parameters should be layer norm.weight/bias, not gamma/beta), can you check whether everything is consistent between train and fine-tune?

ShaobinChen-AH commented 2 years ago

thanks a lot! I will check it.