search

zhihou7 / BatchFormer

CVPR2022, BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning, https://arxiv.org/abs/2203.01522
242 stars 20 forks source link

i can't unserstand the batchformer code in RIDE-LongTailRecognition #4

Open ToscanaGoGithub opened 2 years ago

ToscanaGoGithub commented 2 years ago

Sorry to bother, I'm currently working on the long tail, so I'd like to find the general batchformer code for the long-tailed_recognition task. I can understand the batchformer code part in BalancedSoftmax. But in RIDE-LongTailRecognition, I can't understand what batchformer does, especially lines 137-150 under trainer.py, why is it self.add_bt in [32, 33]? Could you please explain?

zhihou7 commented 2 years ago

Hi, thanks for your questions.

That's a redundant code. It is for debug and ablation studies. From line137-149, the code is actually similar to the code in line150. However, you can set a different loss weight for the stream with BatchFormer and without BatchFormer, which I think is similar to the strategy of reducing the learning rate or increasing weight decay for BatchFormer. Meanwhile, [32,33] includes a shared classifier. Anyway, you can regard this code as redundant and useless code.

The code might include some redundant codes, which show some attempts though we do not use them in the end. However, I think keeping the code might be helpful for readers to better understand the attempts.

By the way, In two-stage RIDE, we first train a model with only BatchFormer on frequent classes or rare classes (see https://github.com/zhihou7/BatchFormer/blob/f6be150ff9fce2ce4bcc3c0f8953dded9d4f273f/long-tailed_recognition/RIDE-LongTailRecognition/model/fb_resnets/RIDEResNet.py#L232). Then, we train the model again with BatchFormer on all classes. WIth a simple attempt, we find distilling with the teacher model (more heads) does not improve the performance further. But we didn't dig it deep.

Feel free to contact me if you have further questions.

ToscanaGoGithub commented 2 years ago

Thanks a lot for the reply! Your reply has solved my problem. This is a very good work, it is worth learning and learning, it may bother you again in the future, thank you again.

zhihou7 commented 2 years ago

Feel free to post questions. Actually, I am also curious about BatchFormer and I am still working on a general version of BatchFormer. Please refer to BatchFormerV2

ToscanaGoGithub commented 2 years ago

In fact, I have noticed that your BatchFormerV2 is very much looking forward to your work, a plug-and-play module is very worth looking forward to.

zhihou7 commented 2 years ago

Thanks! It is really encouraging.