Where is the pretrained detectron2 model?

[kyoto7250]

Thanks for the quick response the other day!

I am now trying to run the visual_grounding directory. I found pretrained bert model in google drive, and yolo v3 model in the script, but I missed pretrained detectron2 model.

I think it's needed to extract features from the bbox. Where is the pretrained detectron2 model?

I'm sorry for asking so many questions 🙏🏻

Best regards.

[Poeroz]

You can download the pretrained detectron2 model via this link.

For more models, you can check the model zoo of detectron2.

[kyoto7250]

@Poeroz

I seem this model can not work.

I changed the name of the model you linked to to "model_weights_path, 'faster_rcnn_R_101_C4_3x.pkl".

# detectron2/model_weights
wget https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/faster_rcnn_R_101_C4_3x/138204752/model_final_298dad.pkl

# rename
mv model_final_298dad.pkl faster_rcnn_R_101_C4_3x.pkl

Then, I run extract_features.py and this error happened.

cd visual_grounding/
python3 extract_features.py

error log

```bash Skip loading parameter 'proposal_generator.rpn_head.conv.weight' to the model due to incompatible shapes: (1024, 1024, 3, 3) in the checkpoint but (256, 256, 3, 3) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.conv.bias' to the model due to incompatible shapes: (1024,) in the checkpoint but (256,) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.objectness_logits.weight' to the model due to incompatible shapes: (15, 1024, 1, 1) in the checkpoint but (3, 256, 1, 1) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.objectness_logits.bias' to the model due to incompatible shapes: (15,) in the checkpoint but (3,) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.anchor_deltas.weight' to the model due to incompatible shapes: (60, 1024, 1, 1) in the checkpoint but (12, 256, 1, 1) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.anchor_deltas.bias' to the model due to incompatible shapes: (60,) in the checkpoint but (12,) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (81, 2048) in the checkpoint but (81, 1024) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (320, 2048) in the checkpoint but (320, 1024) in the model! You might want to double check if this is expected. Some model parameters or buffers are not found in the checkpoint: backbone.bottom_up.res2.0.conv1.norm.{bias, weight} backbone.bottom_up.res2.0.conv1.weight backbone.bottom_up.res2.0.conv2.norm.{bias, weight} backbone.bottom_up.res2.0.conv2.weight backbone.bottom_up.res2.0.conv3.norm.{bias, weight} backbone.bottom_up.res2.0.conv3.weight backbone.bottom_up.res2.0.shortcut.norm.{bias, weight} backbone.bottom_up.res2.0.shortcut.weight backbone.bottom_up.res2.1.conv1.norm.{bias, weight} backbone.bottom_up.res2.1.conv1.weight backbone.bottom_up.res2.1.conv2.norm.{bias, weight} backbone.bottom_up.res2.1.conv2.weight backbone.bottom_up.res2.1.conv3.norm.{bias, weight} backbone.bottom_up.res2.1.conv3.weight backbone.bottom_up.res2.2.conv1.norm.{bias, weight} backbone.bottom_up.res2.2.conv1.weight backbone.bottom_up.res2.2.conv2.norm.{bias, weight} backbone.bottom_up.res2.2.conv2.weight backbone.bottom_up.res2.2.conv3.norm.{bias, weight} backbone.bottom_up.res2.2.conv3.weight backbone.bottom_up.res3.0.conv1.norm.{bias, weight} backbone.bottom_up.res3.0.conv1.weight backbone.bottom_up.res3.0.conv2.norm.{bias, weight} backbone.bottom_up.res3.0.conv2.weight backbone.bottom_up.res3.0.conv3.norm.{bias, weight} backbone.bottom_up.res3.0.conv3.weight backbone.bottom_up.res3.0.shortcut.norm.{bias, weight} backbone.bottom_up.res3.0.shortcut.weight backbone.bottom_up.res3.1.conv1.norm.{bias, weight} backbone.bottom_up.res3.1.conv1.weight backbone.bottom_up.res3.1.conv2.norm.{bias, weight} backbone.bottom_up.res3.1.conv2.weight backbone.bottom_up.res3.1.conv3.norm.{bias, weight} backbone.bottom_up.res3.1.conv3.weight backbone.bottom_up.res3.2.conv1.norm.{bias, weight} backbone.bottom_up.res3.2.conv1.weight backbone.bottom_up.res3.2.conv2.norm.{bias, weight} backbone.bottom_up.res3.2.conv2.weight backbone.bottom_up.res3.2.conv3.norm.{bias, weight} backbone.bottom_up.res3.2.conv3.weight backbone.bottom_up.res3.3.conv1.norm.{bias, weight} backbone.bottom_up.res3.3.conv1.weight backbone.bottom_up.res3.3.conv2.norm.{bias, weight} backbone.bottom_up.res3.3.conv2.weight backbone.bottom_up.res3.3.conv3.norm.{bias, weight} backbone.bottom_up.res3.3.conv3.weight backbone.bottom_up.res4.0.conv1.norm.{bias, weight} backbone.bottom_up.res4.0.conv1.weight backbone.bottom_up.res4.0.conv2.norm.{bias, weight} backbone.bottom_up.res4.0.conv2.weight backbone.bottom_up.res4.0.conv3.norm.{bias, weight} backbone.bottom_up.res4.0.conv3.weight backbone.bottom_up.res4.0.shortcut.norm.{bias, weight} backbone.bottom_up.res4.0.shortcut.weight backbone.bottom_up.res4.1.conv1.norm.{bias, weight} backbone.bottom_up.res4.1.conv1.weight backbone.bottom_up.res4.1.conv2.norm.{bias, weight} backbone.bottom_up.res4.1.conv2.weight backbone.bottom_up.res4.1.conv3.norm.{bias, weight} backbone.bottom_up.res4.1.conv3.weight backbone.bottom_up.res4.2.conv1.norm.{bias, weight} backbone.bottom_up.res4.2.conv1.weight backbone.bottom_up.res4.2.conv2.norm.{bias, weight} backbone.bottom_up.res4.2.conv2.weight backbone.bottom_up.res4.2.conv3.norm.{bias, weight} backbone.bottom_up.res4.2.conv3.weight backbone.bottom_up.res4.3.conv1.norm.{bias, weight} backbone.bottom_up.res4.3.conv1.weight backbone.bottom_up.res4.3.conv2.norm.{bias, weight} backbone.bottom_up.res4.3.conv2.weight backbone.bottom_up.res4.3.conv3.norm.{bias, weight} backbone.bottom_up.res4.3.conv3.weight backbone.bottom_up.res4.4.conv1.norm.{bias, weight} backbone.bottom_up.res4.4.conv1.weight backbone.bottom_up.res4.4.conv2.norm.{bias, weight} backbone.bottom_up.res4.4.conv2.weight backbone.bottom_up.res4.4.conv3.norm.{bias, weight} backbone.bottom_up.res4.4.conv3.weight backbone.bottom_up.res4.5.conv1.norm.{bias, weight} backbone.bottom_up.res4.5.conv1.weight backbone.bottom_up.res4.5.conv2.norm.{bias, weight} backbone.bottom_up.res4.5.conv2.weight backbone.bottom_up.res4.5.conv3.norm.{bias, weight} backbone.bottom_up.res4.5.conv3.weight backbone.bottom_up.res5.0.conv1.norm.{bias, weight} backbone.bottom_up.res5.0.conv1.weight backbone.bottom_up.res5.0.conv2.norm.{bias, weight} backbone.bottom_up.res5.0.conv2.weight backbone.bottom_up.res5.0.conv3.norm.{bias, weight} backbone.bottom_up.res5.0.conv3.weight backbone.bottom_up.res5.0.shortcut.norm.{bias, weight} backbone.bottom_up.res5.0.shortcut.weight backbone.bottom_up.res5.1.conv1.norm.{bias, weight} backbone.bottom_up.res5.1.conv1.weight backbone.bottom_up.res5.1.conv2.norm.{bias, weight} backbone.bottom_up.res5.1.conv2.weight backbone.bottom_up.res5.1.conv3.norm.{bias, weight} backbone.bottom_up.res5.1.conv3.weight backbone.bottom_up.res5.2.conv1.norm.{bias, weight} backbone.bottom_up.res5.2.conv1.weight backbone.bottom_up.res5.2.conv2.norm.{bias, weight} backbone.bottom_up.res5.2.conv2.weight backbone.bottom_up.res5.2.conv3.norm.{bias, weight} backbone.bottom_up.res5.2.conv3.weight backbone.bottom_up.stem.conv1.norm.{bias, weight} backbone.bottom_up.stem.conv1.weight backbone.fpn_lateral2.{bias, weight} backbone.fpn_lateral3.{bias, weight} backbone.fpn_lateral4.{bias, weight} backbone.fpn_lateral5.{bias, weight} backbone.fpn_output2.{bias, weight} backbone.fpn_output3.{bias, weight} backbone.fpn_output4.{bias, weight} backbone.fpn_output5.{bias, weight} proposal_generator.rpn_head.anchor_deltas.{bias, weight} proposal_generator.rpn_head.conv.{bias, weight} proposal_generator.rpn_head.objectness_logits.{bias, weight} roi_heads.box_head.fc1.{bias, weight} roi_heads.box_head.fc2.{bias, weight} roi_heads.box_predictor.bbox_pred.weight roi_heads.box_predictor.cls_score.weight roi_heads.mask_head.deconv.{bias, weight} roi_heads.mask_head.mask_fcn1.{bias, weight} roi_heads.mask_head.mask_fcn2.{bias, weight} roi_heads.mask_head.mask_fcn3.{bias, weight} roi_heads.mask_head.mask_fcn4.{bias, weight} roi_heads.mask_head.predictor.{bias, weight} The checkpoint state_dict contains keys that are not used by the model: backbone.stem.conv1.weight backbone.stem.conv1.norm.{bias, running_mean, running_var, weight} backbone.res2.0.shortcut.weight backbone.res2.0.shortcut.norm.{bias, running_mean, running_var, weight} backbone.res2.0.conv1.weight backbone.res2.0.conv1.norm.{bias, running_mean, running_var, weight} backbone.res2.0.conv2.weight backbone.res2.0.conv2.norm.{bias, running_mean, running_var, weight} backbone.res2.0.conv3.weight backbone.res2.0.conv3.norm.{bias, running_mean, running_var, weight} backbone.res2.1.conv1.weight backbone.res2.1.conv1.norm.{bias, running_mean, running_var, weight} backbone.res2.1.conv2.weight backbone.res2.1.conv2.norm.{bias, running_mean, running_var, weight} backbone.res2.1.conv3.weight backbone.res2.1.conv3.norm.{bias, running_mean, running_var, weight} backbone.res2.2.conv1.weight backbone.res2.2.conv1.norm.{bias, running_mean, running_var, weight} backbone.res2.2.conv2.weight backbone.res2.2.conv2.norm.{bias, running_mean, running_var, weight} backbone.res2.2.conv3.weight backbone.res2.2.conv3.norm.{bias, running_mean, running_var, weight} backbone.res3.0.shortcut.weight backbone.res3.0.shortcut.norm.{bias, running_mean, running_var, weight} backbone.res3.0.conv1.weight backbone.res3.0.conv1.norm.{bias, running_mean, running_var, weight} backbone.res3.0.conv2.weight backbone.res3.0.conv2.norm.{bias, running_mean, running_var, weight} backbone.res3.0.conv3.weight backbone.res3.0.conv3.norm.{bias, running_mean, running_var, weight} backbone.res3.1.conv1.weight backbone.res3.1.conv1.norm.{bias, running_mean, running_var, weight} backbone.res3.1.conv2.weight backbone.res3.1.conv2.norm.{bias, running_mean, running_var, weight} backbone.res3.1.conv3.weight backbone.res3.1.conv3.norm.{bias, running_mean, running_var, weight} backbone.res3.2.conv1.weight backbone.res3.2.conv1.norm.{bias, running_mean, running_var, weight} backbone.res3.2.conv2.weight backbone.res3.2.conv2.norm.{bias, running_mean, running_var, weight} backbone.res3.2.conv3.weight backbone.res3.2.conv3.norm.{bias, running_mean, running_var, weight} backbone.res3.3.conv1.weight backbone.res3.3.conv1.norm.{bias, running_mean, running_var, weight} backbone.res3.3.conv2.weight backbone.res3.3.conv2.norm.{bias, running_mean, running_var, weight} backbone.res3.3.conv3.weight backbone.res3.3.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.0.shortcut.weight backbone.res4.0.shortcut.norm.{bias, running_mean, running_var, weight} backbone.res4.0.conv1.weight backbone.res4.0.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.0.conv2.weight backbone.res4.0.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.0.conv3.weight backbone.res4.0.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.1.conv1.weight backbone.res4.1.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.1.conv2.weight backbone.res4.1.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.1.conv3.weight backbone.res4.1.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.2.conv1.weight backbone.res4.2.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.2.conv2.weight backbone.res4.2.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.2.conv3.weight backbone.res4.2.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.3.conv1.weight backbone.res4.3.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.3.conv2.weight backbone.res4.3.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.3.conv3.weight backbone.res4.3.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.4.conv1.weight backbone.res4.4.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.4.conv2.weight backbone.res4.4.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.4.conv3.weight backbone.res4.4.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.5.conv1.weight backbone.res4.5.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.5.conv2.weight backbone.res4.5.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.5.conv3.weight backbone.res4.5.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.6.conv1.weight backbone.res4.6.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.6.conv2.weight backbone.res4.6.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.6.conv3.weight backbone.res4.6.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.7.conv1.weight backbone.res4.7.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.7.conv2.weight backbone.res4.7.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.7.conv3.weight backbone.res4.7.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.8.conv1.weight backbone.res4.8.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.8.conv2.weight backbone.res4.8.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.8.conv3.weight backbone.res4.8.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.9.conv1.weight backbone.res4.9.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.9.conv2.weight backbone.res4.9.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.9.conv3.weight backbone.res4.9.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.10.conv1.weight backbone.res4.10.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.10.conv2.weight backbone.res4.10.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.10.conv3.weight backbone.res4.10.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.11.conv1.weight backbone.res4.11.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.11.conv2.weight backbone.res4.11.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.11.conv3.weight backbone.res4.11.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.12.conv1.weight backbone.res4.12.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.12.conv2.weight backbone.res4.12.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.12.conv3.weight backbone.res4.12.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.13.conv1.weight backbone.res4.13.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.13.conv2.weight backbone.res4.13.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.13.conv3.weight backbone.res4.13.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.14.conv1.weight backbone.res4.14.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.14.conv2.weight backbone.res4.14.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.14.conv3.weight backbone.res4.14.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.15.conv1.weight backbone.res4.15.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.15.conv2.weight backbone.res4.15.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.15.conv3.weight backbone.res4.15.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.16.conv1.weight backbone.res4.16.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.16.conv2.weight backbone.res4.16.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.16.conv3.weight backbone.res4.16.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.17.conv1.weight backbone.res4.17.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.17.conv2.weight backbone.res4.17.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.17.conv3.weight backbone.res4.17.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.18.conv1.weight backbone.res4.18.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.18.conv2.weight backbone.res4.18.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.18.conv3.weight backbone.res4.18.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.19.conv1.weight backbone.res4.19.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.19.conv2.weight backbone.res4.19.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.19.conv3.weight backbone.res4.19.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.20.conv1.weight backbone.res4.20.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.20.conv2.weight backbone.res4.20.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.20.conv3.weight backbone.res4.20.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.21.conv1.weight backbone.res4.21.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.21.conv2.weight backbone.res4.21.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.21.conv3.weight backbone.res4.21.conv3.norm.{bias, running_mean, running_var, weight} backbone.res4.22.conv1.weight backbone.res4.22.conv1.norm.{bias, running_mean, running_var, weight} backbone.res4.22.conv2.weight backbone.res4.22.conv2.norm.{bias, running_mean, running_var, weight} backbone.res4.22.conv3.weight backbone.res4.22.conv3.norm.{bias, running_mean, running_var, weight} roi_heads.res5.0.shortcut.weight roi_heads.res5.0.shortcut.norm.{bias, running_mean, running_var, weight} roi_heads.res5.0.conv1.weight roi_heads.res5.0.conv1.norm.{bias, running_mean, running_var, weight} roi_heads.res5.0.conv2.weight roi_heads.res5.0.conv2.norm.{bias, running_mean, running_var, weight} roi_heads.res5.0.conv3.weight roi_heads.res5.0.conv3.norm.{bias, running_mean, running_var, weight} roi_heads.res5.1.conv1.weight roi_heads.res5.1.conv1.norm.{bias, running_mean, running_var, weight} roi_heads.res5.1.conv2.weight roi_heads.res5.1.conv2.norm.{bias, running_mean, running_var, weight} roi_heads.res5.1.conv3.weight roi_heads.res5.1.conv3.norm.{bias, running_mean, running_var, weight} roi_heads.res5.2.conv1.weight roi_heads.res5.2.conv1.norm.{bias, running_mean, running_var, weight} roi_heads.res5.2.conv2.weight roi_heads.res5.2.conv2.norm.{bias, running_mean, running_var, weight} roi_heads.res5.2.conv3.weight roi_heads.res5.2.conv3.norm.{bias, running_mean, running_var, weight} Traceback (most recent call last): File "/home/kyoto7250/PLUVR/visual_grounding/extract_features.py", line 139, in main() File "/home/kyoto7250/PLUVR/visual_grounding/extract_features.py", line 106, in main model.roi_heads.res5[2].conv3.register_forward_hook(hook_fn_forward) File "/home/kyoto7250/.pyenv/versions/3.9.12/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1177, in __getattr__ raise AttributeError("'{}' object has no attribute '{}'".format( AttributeError: 'StandardROIHeads' object has no attribute 'res5' ```

Is these the correct the model? It would be helpful if you could also tell us your version of detectron2🙏🏻

my environment

• detectron2 0.6+cu113
• python3.9. 13 12
• pytorch-pretrained-bert 0.6.2
• torch 1.10.1+cu113
• torchvision 0.11.2+cu113

Thank you.

[kyoto7250]

And config file is here

P.S It seems that config and load specify different models.

[kyoto7250]

@Poeroz

friendly mention :)

Would you be willing to share your opinion? I can't reproduce it as it is...

[Gouryella]

Change 'extract_features.py' file line 49 to '_default="detectron2/configs/COCO-InstanceSegmentation/mask_rcnn_R_101_C43x.yaml",'