Please add OHEM - Githubissues

neoml-lib / neoml

Machine learning framework for both deep learning and traditional algorithms

https://www.abbyy.com/neoml/

Apache License 2.0

766 stars 126 forks source link

Please add OHEM #454

Open MaratKhabibullin opened 3 years ago

MaratKhabibullin commented 3 years ago

Please add online hard example mining (OHEM) based on BCE, L1, L2 and other losses for binary classification.

Loss = SumOfPositiveClassLosses / + alpha SumOfNegativeClassLosses / + beta SumOfHARDNegativeClassLosses / <number of hard negative class examples>

hard negatives are top K negative class elements with highest loss value.

alpha, beta, K are external parameters.

FedyuninV commented 2 years ago

Can you please share the implementation of this feature (or at least some paper). Because google shows that this feature is implemented differently. E.g. I've found an implementation (also OHEM) which just back propagates only top-K losses.

ps plz, fix your formula, I just don't get what SumOfPositiveClassLosses / + alpha means...