search

BloodAxe / pytorch-toolbelt

PyTorch extensions for fast R&D prototyping and Kaggle farming
MIT License
1.51k stars 119 forks source link

Source on BalancedBCEWithLogitsLoss #65

Closed davidhuangal closed 2 years ago

davidhuangal commented 2 years ago

I was wondering if there is a paper or even just a description on the BalancedBCEWithLogitsLoss ?

BloodAxe commented 2 years ago

Hi! Indeed, there are. See https://arxiv.org/pdf/1504.06375.pdf formula 2. Will update the code for the future release with proper docs.

davidhuangal commented 2 years ago

Thank you very much!

davidhuangal commented 2 years ago

I have two more questions about this loss function: what does the parameter gamma correspond with in formula 2 since there isn't a gamma symbol in it, and how does it effect the loss?

BloodAxe commented 2 years ago

The gamma is a my addition to this loss, when gamma=1, it's equal to vanilla BBCE as in Formula 2. Values >1 decay the balancing weights. E.g if gamma = 2, and positives fraction is 0.25 and negatives is 0.75, corresponding weights will be 0.25^2 and 0.75^ accordingly.

davidhuangal commented 2 years ago

Got it, that makes sense. Thank you again.