Open atuleu opened 1 year ago
Gotcha, so do mean and sum in focal loss provide different results?
I think we should just let it accept Reduction
similar to other keras losses? @atuleu
@atuleu , Could you please make the required changes as per the comment in the linked PR. Thanks
Short Description
Currently
FocalLoss
reduces multi - label problems by amean
(see below):https://github.com/keras-team/keras-cv/blob/ff20079aebe8e80bc355c279c9c5541a7c823535/keras_cv/losses/focal.py#L90
However a strictly correct formulation would be a
sum
. Original implementaion ( Facebook Research, pytorch ) lets the user decide for asum
or amean
. Other usage of focal loss also introduces a sum over non-zero elements scheme (see below).I think the API should either:
sum
,mean
orsum_over_non_zero
class reductionPlease Note that it is fundamentaly different from the
reduction
parameter of keras losses that applies primarily on batches. Here we are talking on howFocalLoss
, a modified binary cross-entropy loss, should be extended from single class to multi-class, multi-label cases.What do you think @LukeWood @quantumalaviya ?
Papers
Existing Implementations
Other Information
First discussed in #672