Question about model convergence

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

MIT License

3.33k stars 505 forks source link

Question about model convergence #193

Closed Jessica-jia closed 1 year ago

Jessica-jia commented 1 year ago

How can I tell if the model is converging or not, and is the value of the loss function indicative? I trained on my own dataset and the loss of each iteration is around 5.0, is this correct? Thanks to the author for sharing the code!

gaotianyu1350 commented 1 year ago

Hi,

Loss around 5.0 seems to be too high. Usually for unsupervised datasets the loss should converge to almost 0; for supervised ones it should also be pretty low.