Open ZhuYun97 opened 3 years ago
Hi @ZhuYun97,
Thank you for asking. Please see the in the paper https://arxiv.org/pdf/2010.13902.pdf eq. (3), and also refer to the eq. (1) in simclr paper http://proceedings.mlr.press/v119/chen20j/chen20j.pdf.
Thanks for your reply. But I still have some questions. The denominator contains all pairs(both positive and negative) in simclr(eq. 1). But in graphcl(eq. 3), the denominator does not contain positive pair. I am confused why the denominator of contrastive loss in graphcl doesn't contain positive pair. Does the accuracy perform better than the contrastive loss which contains positive pair in the denominator?
Numerically I did not find a difference.
Thanks for your awesome codes, but I have some questions about![\ell_{i, j}=-\log \frac{\exp \left(\operatorname{sim}\left(\boldsymbol{z}_{i}, \boldsymbol{z}_{j}\right) / \tau\right)}{\sum_{k=1}^{N} \exp \left(\operatorname{sim}\left(\boldsymbol{z}_{i}, \boldsymbol{z}_{k}\right) / \tau\right)}](https://latex.codecogs.com/gif.latex?\ell_{i,&space;j}=-\log&space;\frac{\exp&space;\left(\operatorname{sim}\left(\boldsymbol{z}_{i},&space;\boldsymbol{z}_{j}\right)&space;/&space;\tau\right)}{\sum_{k=1}^{N}&space;\exp&space;\left(\operatorname{sim}\left(\boldsymbol{z}_{i},&space;\boldsymbol{z}_{k}\right)&space;/&space;\tau\right)})
./unsupervised_TU/gsimclr.py
file. In theloss_cal
function, the denominator will minus the positive pairloss = pos_sim / (sim_matrix.sum(dim=1) - pos_sim)
which is not consistent with infonce loss