MoCo implements

[handsomelys]

Can you demonstrate how to use dcl in MoCo? I'm a bit puzzled because after I replaced infoNCE loss with your DCL code, the accuracy of the training did not improve as the loss decreased.

[enze5088]

Can you demonstrate how to use dcl in MoCo? I'm a bit puzzled because after I replaced infoNCE loss with your DCL code, the accuracy of the training did not improve as the loss decreased.

I have the same phenomenon in the datasets which I used. And indeed it may not necessarily be better than infoNCE in some experiments.

[handsomelys]

thanks for reply~

[handsomelys]

Hello, I found that using InfoNCE in your code (the one implemented without cross-entropy) does not converge in my code like DCL, but the InfoNCE implemented with cross-entropy does converge, why? (Experiment was run on MOCO)

[enze5088]

Hello, I found that using InfoNCE in your code (the one implemented without cross-entropy) does not converge in my code like DCL, but the InfoNCE implemented with cross-entropy does converge, why? (Experiment was run on MOCO)

@handsomelys

一个是直接令正样本相似度趋近于1，负样本相似度趋近于0.一个是交叉熵优化。前者图像中的对比学习用的更多一些。

[handsomelys]

可能我没表达清楚，就是我把你代码中的infoNCE的实现拿去MoCo那里用，然后发现不使用交叉熵的那种实现方式收敛不了，使用交叉熵实现的那个infoNCE就收敛得了，不知道你有没有遇到过这种情况，能给我一点分析的建议吗。

[enze5088]

可能我没表达清楚，就是我把你代码中的infoNCE的实现拿去MoCo那里用，然后发现不使用交叉熵的那种实现方式收敛不了，使用交叉熵实现的那个infoNCE就收敛得了，不知道你有没有遇到过这种情况，能给我一点分析的建议吗。

这个可能跟数据集和任务类型有关，我也并不清楚应该怎么解释，只是猜想交叉熵可能放松了约束。