search

zylo117 / Yet-Another-EfficientDet-Pytorch

The pytorch re-implement of the official efficientdet with SOTA performance in real time and pretrained weights.
GNU Lesser General Public License v3.0
5.2k stars 1.27k forks source link

BCE calculation in FocalLoss can output NaN values #717

Open dav-ell opened 2 years ago

dav-ell commented 2 years ago

The BCE calculation in FocalLoss in this repo is missing the log clamp to [-100, inf]

The current calculation is this:

https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/15403b5371a64defb2a7c74e162c6e880a7f462c/efficientdet/loss.py#L115

The official implementation of BCELoss in PyTorch states this:

Our solution [to possible NaNs] is that BCELoss clamps its log function outputs to be greater than or equal to -100. This way, we can always have a finite loss value and a linear backward method.

The correct implementation is following, which I've verified fixes the NaN problem:

bce = -(targets * torch.clamp(torch.log(classification), min=-100) + (1.0 - targets) * torch.clamp(torch.log(1.0 - classification), min=-100))
dav-ell commented 2 years ago

@zylo117 are you planning on making further commits to this repo, or should someone fork this and apply updates elsewhere?

santhoshnumberone commented 2 years ago

@dav-ell Correcting this and training from scratch will have a better mAP value overall closer to official implementation?