search

hustvl / MIMDet

[ICCV 2023] You Only Look at One Partial Sequence
https://arxiv.org/abs/2204.02964
MIT License
336 stars 31 forks source link

MIM training while training object detection #10

Closed bamps53 closed 2 years ago

bamps53 commented 2 years ago

Hi, thanks for sharing such a interesting work! Have you tried to train decoder with reconstruction loss while training object detection part? I know there is a kind of information leakage by changing image patcher to conv one. But same thing can be said to current modeling(i.e. MIM is not useful with conv patchifier), but it seems working well and improving object detection performance. So my question is, have you already tried and found that taking reconstruction loss actually degrade the benefit for object detection part?

Thanks in advance!

Yuxin-CV commented 2 years ago

Hi @bamps53 and thank you for your interest in our work.

We are trying a similar approach, and we will update in our revision as long as the recon objective is beneficial to detection performances.

bamps53 commented 2 years ago

Thanks for the answer! Good to know you're trying same idea, can't wait to see the results:) It means currently it's not clear whether it's helpful or not, right? I'll also try it in my project.

Thanks!