dvlab-research / ReviewKD

Distilling Knowledge via Knowledge Review, CVPR 2021
248 stars 34 forks source link

Questions about detection pretrained weights #7

Closed Coldfire93 closed 3 years ago

Coldfire93 commented 3 years ago

I want to make sure that the file mv2-r50.pth in the detection pretrained weights you provided contains both teacher's and student's weights.

Thank you!

akuxcw commented 3 years ago

Yes, I just checked the mv2-r50.pth contains both teacher and student.

Coldfire93 commented 3 years ago

Hi @akuxcw , The size of the mv2-r50.pth is 266MB. The teacher model is about 160MB ( I'm not sure. I used torchvision's pretrained weights before. It is about 160MB. Your teacher's weights come from Detectron2's pretrained detector. Was there a particular reason why you chose to use Detectron2 instead of torchvision?). So, the student model's size is about 100MB? Is that right? Thank you!

akuxcw commented 3 years ago

We use the teacher model from Detectron2 because it provides pretrained detectors and we train our student based on Detectron2. I think there are no pretrained detectors in torchvision, torchvision only provides pretrained weights on ImageNet. Which model do you use before? And we use this model as teacher, which is about 230M, so student's model is about 30M.

Coldfire93 commented 3 years ago

Hi @akuxcw , We used this model as teacher. It provides detectors too.

And we use this model as teacher, which is about 230M

Is the backbone of the link you gave above ResNet101? Could you please send me the link of model with ResNet50 as backbone? Because the mv2-r50.pth - teacher model's backbone is ResNet50.

Thank you!

akuxcw commented 3 years ago

Sorry, faster-rcnn-50 is available here. And you are right about the model size. I really don't know torchvision provides pretrained detectors. Thank you for telling me that! And we use detectron2's pretrained weights just because we implement our method based on detectron2.

Coldfire93 commented 3 years ago

Hi @akuxcw , I see. Thank you~

twmht commented 2 years ago

@akuxcw

Did you also switch teacher to training mode for BN when training detector?