search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.05k stars 2.45k forks source link

Add on losses #1374

Open arita37 opened 2 years ago

arita37 commented 2 years ago

Hello,

do you think we can add this loss ?

https://arxiv.org/abs/2103.13538

it works well on large dataset when there are some implicit hiearchy. (ie news category,…)

nreimers commented 2 years ago

Yes, looks really interesting.

Do you know if code is available? Would you be able to create a pull request?

julianStreibel commented 2 years ago

It seems like there is no code available (https://paperswithcode.com/paper/hierarchical-proxy-based-loss-for-deep-metric). There would also be a need for some "basic" proxy loss like Proxy-NCA and Proxy Anchor (https://github.com/KevinMusgrave/pytorch-metric-learning/blob/master/src/pytorch_metric_learning/losses/proxy_losses.py, https://github.com/KevinMusgrave/pytorch-metric-learning/blob/master/src/pytorch_metric_learning/losses/proxy_anchor_loss.py).

julianStreibel commented 2 years ago

Im working on the implementation and im questioning where you would use this loss in NLP tasks. Im getting the standard case where we have 20 classes and want to classify the sentences on these classes. But it does not really scale to tasks like NLI right?