search

PRBonn / segcontrast

MIT License
96 stars 13 forks source link

In what part of the code ContrastiveLoss is used? #18

Closed Rezamh13 closed 1 year ago

Rezamh13 commented 1 year ago

Hi, I was browsing through the code, in contrastive_train.py where you imported ContrastiveLoss but set criterion = nn.CrossEntropyLoss().cuda(). Could you explain where exactly ContrastiveLoss is used?

nuneslu commented 1 year ago

Hi, this is because we used the contrastive loss formulation from MoCo paper. They "simplify" the loss to a cross-entropy function. In the end is the same, but it simplifies the similarity computation between positives and negatives.

Rezamh13 commented 1 year ago

@nuneslu Could you share the pre-training loss curve?

nuneslu commented 1 year ago

Hi, yes, this is the loss curve: image

But note that in the beginning is normal for the loss to increase a bit since it is still accumulating segments in the feature bank: image

nuneslu commented 1 year ago

Also, in case you are interested, we have just released the code for our new pre-training method here https://github.com/PRBonn/TARL

nuneslu commented 1 year ago

I will close this now. If you have any more questions, feel free to reopen it.