Add MultipleNegativesRanking Loss

KevinMusgrave / pytorch-metric-learning

The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.

https://kevinmusgrave.github.io/pytorch-metric-learning/

MIT License

6.02k stars 658 forks source link

Add MultipleNegativesRanking Loss #664

Closed mcschmitz closed 11 months ago

mcschmitz commented 1 year ago

Hi all!

I'd be great if we could add MultipleNegativesRankingLoss as used in the sentece bert model and introduced in Efficient Natural Language Response Suggestion for Smart Reply. I can provide a PR if you're interested.

KevinMusgrave commented 1 year ago

A PR would be very much appreciated!

KevinMusgrave commented 11 months ago

As mentioned in the pull request, I think this is equivalent to NTXentLoss with a customized reducer:

from pytorch_metric_learning.losses import NTXentLoss
from pytorch_metric_learning.reducers import AvgNonZeroReducer, PerAnchorReducer

loss_fn = NTXentLoss(reducer=PerAnchorReducer(AvgNonZeroReducer()))