How to add focal loss for YOLOv2 and resnet50 in darknet？

[TaihuLight]

Focal loss is a method to process the imbalance examples. it is not better in the balance scene. for the level of imbalance , choose alpha=0.5 and gamma=2 may improve by 2% according to the author. However, according to the review about the implementation of focal loss in darknet: http://blog.csdn.net/linmingan/article/details/77885832 his implementation may be wrong. another implementation like https://github.com/unsky/focal-loss#now-focal-loss-with-softmax-work-well or https://github.com/clcarwin/focal_loss_pytorch
may be help for implementing focal loss in darknet to get the better mAP.

[AlexeyAB]

• You could not change the source code?
• Or did you implement focal-loss, but it gives you low mAP (precision)?

I added code from your link, but I didn't test it: http://blog.csdn.net/linmingan/article/details/77885832 To use this implementation of focal-loss, just update your cod from this repo and add focal_loss=1 in the [region] layer in your cfg-file: https://github.com/AlexeyAB/darknet/blob/5e448c00f32e80e13eead72318047ce8368aac9a/cfg/yolo-voc.2.0.cfg#L236

So you can try to fine-tune ResNet-152 with focal_loss=1 which I added: https://github.com/AlexeyAB/darknet/blob/master/build/darknet/x64/resnet152_yolo.cfg

• Just download: https://pjreddie.com/media/files/resnet152.weights
• do darknet.exe partial cfg/resnet152.cfg resnet152.weights resnet152.201 201
• train darknet.exe detector train voc.data resnet152_yolo.cfg resnet152.201

The model will be fine-tuned starting from this layer: https://github.com/AlexeyAB/darknet/blob/0c07837397e791c750e81857e214d315ea24199f/build/darknet/x64/resnet152_yolo.cfg#L1431-L1434

[AlexeyAB]

Implementation uses: alpha=0.5, gamma=2

Added code: https://github.com/AlexeyAB/darknet/blob/5e448c00f32e80e13eead72318047ce8368aac9a/src/region_layer.c#L133-L146

In general it should increase mAP, but may be just using focal-loss is not enough to increase mAP. You can see all about RetinaNet: https://arxiv.org/abs/1708.02002v2

As said for RetinaNet in Box - Dev: http://cocodataset.org/#detections-leaderboard

RetinaNet is a dense one-stage object detector trained with focal loss to address the extreme class imbalance in training examples [1]. The model was fine-tuned with a ResNet-101 pretrained model on ImageNet-1k. This entry reports single model accuracy with vanilla training and testing procedures without any bells and whistles and runs at 5FPS. [1] Focal Loss for Dense Object Detection

[MyVanitar]

This focal loss only can be applied to the ResNet? can we test it with Darknet-19?

[AlexeyAB]

@VanitarNordic It can be applied to any Yolo networks. In the article they got optimal (precision-speed) result on ResNet-101.

[MyVanitar]

@AlexeyAB

This focal loss is something like making hard examples to make a better weights?

By the way we can just pas your added CFG parameter and train and see if it will increase the performance, Yes?

I briefly read the paper, mAP increase was fantastic!

[AlexeyAB]

@VanitarNordic Yes, just add focal_loss=1 to the [region] layer in any yolo cfg-file. And update code from this repo.

Also if it leads to Nan, then decrease this value to 0.25 https://github.com/AlexeyAB/darknet/blob/5e448c00f32e80e13eead72318047ce8368aac9a/src/region_layer.c#L135

But I didn't test focal_loss yet. And I did not check whether this is the correct implementation: http://blog.csdn.net/linmingan/article/details/77885832

[AlexeyAB]

Results of Focal Loss with Yolo for VOC2007-test dataset: https://github.com/marvis/pytorch-yolo2#focal-loss

But sometimes, when there aren't hard examples, then using FL (Focal-loss) people get wrose result than using CE (Cross entropy) or OHEM (Online Hard Example Mining): https://github.com/unsky/focal-loss/issues/5#issuecomment-343841216

At first glance it looks like there is a slightly different implementation here https://github.com/unsky/focal-loss#now-focal-loss-with-softmax-work-well

---

About this:

This focal loss is something like making hard examples to make a better weights?

Yes!

[MyVanitar]

I trained with focal_loss=1

I did not face any nan problem and I trained till the end for one class and 2,250 iterations.

I can not say it has improved or decreased the performance because before applying this update I was getting FP = 0 and FN = 1, but after this update I get FP=1 and FN=0. IoU reduced about 1%.

it worth to investigate more on a bigger dataset with higher number of classes.

[litingsjj]

@AlexeyAB Have you tried OHEM (Online Hard Example Mining) in yolo? When I choose negative sample randomly. The background is studied not well.

[AlexeyAB]

@litingsjj No, I haven't.

[spaul13]

How to apply focal loss for a classification model (in the cfg file), if I don't have the yolo layer?

If I can't then can I include yolo layer before the last layer for any classification model to add the focal loss?