should I add parameters of InfoNCE to optimizer for optimizing?

[xinyaojiejie]

I use pip install the infoGAN-pytorch, I know Mutual information is estimated by network. And when I use the code, should I add the parameters of class InfoNCE to the optimizer for optimizing, or I can directly use the function and it will give me the correct loss? Thank you!

[RElbers]

Hello. Sorry for the late reply. InfoNCE loss has no parameters, so you don't need to add it the the optimizer.

[xinyaojiejie]

thank you for your reply. I also want to ask that I noticed that the function also can only accept positive samples, and I know that to calculate infonce loss it need large amount of negative samples, and if there is no negative samples are fed, is it also can get the accurate calculation of infonce or you use the method that no need negative samples?

RElbers @.***> 于2023年4月14日周五 22:44写道：

Hello. Sorry for the late reply. InfoNCE loss has no parameters, so you don't need to add it the the optimizer.

— Reply to this email directly, view it on GitHub https://github.com/RElbers/info-nce-pytorch/issues/11#issuecomment-1508450862, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKVJQ5IGSUW44Y3KA6LLEDXBFBCRANCNFSM6AAAAAAWONLLAY . You are receiving this because you authored the thread.Message ID: @.***>

[RElbers]

The negative samples are calculated from the batch of positive samples Say you have a batch of N pairs of positive and query samples. Then the similarity between all combinations of positive and query is calculated, giving a N by N matrix. The diagonal of this matrix have the values for the positive pairs. The values outside of the diagonal are considered the negative samples. This means that to have enough negative samples, the batch-size also needs to be large.