keras-team / keras-cv

Industry-strength Computer Vision workflows with Keras
Other
988 stars 321 forks source link

Adds different class reduction scheme in `FocalLoss` #681

Open atuleu opened 1 year ago

atuleu commented 1 year ago

Short Description

Currently FocalLoss reduces multi - label problems by a mean (see below):

https://github.com/keras-team/keras-cv/blob/ff20079aebe8e80bc355c279c9c5541a7c823535/keras_cv/losses/focal.py#L90

However a strictly correct formulation would be a sum. Original implementaion ( Facebook Research, pytorch ) lets the user decide for a sum or a mean. Other usage of focal loss also introduces a sum over non-zero elements scheme (see below).

I think the API should either:

Please Note that it is fundamentaly different from the reduction parameter of keras losses that applies primarily on batches. Here we are talking on how FocalLoss, a modified binary cross-entropy loss, should be extended from single class to multi-class, multi-label cases.

What do you think @LukeWood @quantumalaviya ?

Papers

Existing Implementations

Other Information

First discussed in #672

LukeWood commented 1 year ago

Gotcha, so do mean and sum in focal loss provide different results?

tanzhenyu commented 1 year ago

I think we should just let it accept Reduction similar to other keras losses? @atuleu

sachinprasadhs commented 6 months ago

@atuleu , Could you please make the required changes as per the comment in the linked PR. Thanks