search

megvii-research / BBN

The official PyTorch implementation of paper BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition
https://arxiv.org/abs/1912.02413
MIT License
657 stars 100 forks source link

question about loss curve at inaturalist2019 #3

Closed Haijunlv closed 4 years ago

Haijunlv commented 4 years ago

hi, i just use bbn to train in inaturalist2019, the config is same as config of iNaturalist2018.yaml. thanks to the clean code, it is very easy to get good result with resnet50 baseline 2x. But i found loss curve is very strage. train loss at 67 epoch got highest. Have you gotten the same curve? Could you explain the reason?

image

Haijunlv commented 4 years ago

result on exapmle cifar10 also show same cure image

ZhouBoyan commented 4 years ago

Yes, I have got the same curve as you. Acctually, two branches take care of head class and tail class respectively, which indicates that the conventional branch tends to classify one picture as head class and the re-balancing branch as tail class. With the cumulative learning strategy, the learning focus gradually moves from the conventional branch to the re-balancing branch, leading to the two branches conflicting with each other (e.g. epoch 60-80) . With the loss optimized, the conflicts gradually disappeared. We advise you to split the combined fc (I combine them just for simplification) into two parts （one for conventional branch and the other for re-balancing branch）and observe their individual accuracy or loss.