Implemented NCE loss. - Githubissues

RElbers / info-nce-pytorch

PyTorch implementation of the InfoNCE loss for self-supervised learning.

MIT License

445 stars 37 forks source link

Implemented NCE loss. #17

Closed k920049 closed 8 months ago

k920049 commented 8 months ago

Describe your changes

[v] Implemented NCE loss with log sum exponential techniques.
[v] Added some test cases.

k920049 commented 8 months ago

After carefully examining the Cross Entropy loss, I realized that the function indeed does not ignore off diagonal components. Thanks for the suggestion and carefully testing the results.