Varifocal loss is increasing.

[XuyangBai]

Hi, thanks for sharing this excellent work. When I add an IoU prediction branch with varifocal loss to my anchor-free detection model, the mAP does increase, but I find the value of varifocal loss only decreases for the first several hundred iterations and then keeps increasing during the whole training process. I suspect the reason is that at the beginning the iou of all predicted bboxes are very small thus the network can predict 0 for all the bbox for a small loss value. Then with the training, the iou becomes larger and more diverse, so the error between predicted iou and gt iou might also increase. But it is still hard to understand why the loss term contributes to the final performance given that the values keep increasing during training. Do you have any similar observations or explanations?

[hyz-xmaster]

Hi, I guess there might be something wrong with your training. Please have a look at those training logs I shared on the webpage, like this one. You can see that the loss_cls term has a clear downward trend during the whole training.

In fact, we merge the IoU prediction into the classification score, rather than predict a single IoU value.

[XuyangBai]

Yes I know that the varifocal loss is proposed to predict a class-aware iou value but I think it is also able to use for the seperated iou prediction branch (so the network predict a iou value for each class and the varifocal loss is used to replace the CE loss to handle class imbalance problem). And I also suspect my there might be something wrong with my training, but the mAP did increase during the training and adding the iou prediction branch with varifocal loss also improve the results. The only wired thing is that the varifocal loss is increasing.