Temperature v.s. Base Temperature

sayakpaul / Supervised-Contrastive-Learning-in-TensorFlow-2

Implements the ideas presented in https://arxiv.org/pdf/2004.11362v1.pdf by Khosla et al.

https://app.wandb.ai/authors/scl/reports/Improving-Image-Classification-with-Supervised-Contrastive-Learning--VmlldzoxMzQwNzE

132 stars 24 forks source link

Temperature v.s. Base Temperature #3

Closed mmderakhshani closed 4 years ago

mmderakhshani commented 4 years ago

Hi,

Thanks for providing your code. I have read your loss function and I found that there are two temperatures (temperature and base_temperate). could you please tell me what is the difference between these two parameters? I am looking for the effect of base_temperature.

Thanks.

sayakpaul commented 4 years ago

Hey.

We use max_margin_contrastive_loss instead of the supervised NT-Xent loss in our code as it's a bit faster to compute. Regarding your original question, base_temperature acts as another scaling parameter to further push the original temperature as per my understanding. I would also suggest reaching out to Yonglong (yonglong@mit.edu) (who authored the loss) for further details.