search

fudan-zvg / DeepInteraction

[NeurIPS 2022] DeepInteraction: 3D Object Detection via Modality Interaction
MIT License
221 stars 16 forks source link

About Image backbone checkpoint #15

Open sunnyHelen opened 1 year ago

sunnyHelen commented 1 year ago

Hi, thanks a lot for sharing your great work. I want to know where to find the pretrained R50 on COCO and NuImage for the initialization of the backbone of the camera branch.

Alexander0Yang commented 1 year ago

The pretrained image backbone weights can be downloaded from https://download.openmmlab.com/mmdetection3d/v0.1.0_models/nuimages_semseg/cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim/cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim_20201009_124951-40963960.pth. Or you can train it by yourself using the corresponding config.

VijayLuo commented 1 year ago

Hi, awesome work! the DeepInteraction-large uses Swin-Tiny as image backbone. but i didn't find the pretrained image backbone weights in the paper and this repository. Where can i download it or how can i train it.

MaiRajborirug commented 1 year ago

Hi, thank you for providing everything necessary

@Alexander0Yang I wonder in the tools/test.py code, which checkpoint should I use? There are r50_fpn_voxel.pth, Fusion_0075_refactor.pth, and the one you mention earlier cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim_20201009_124951-40963960.pth.

From my understanding Fusion_0075_refactor.pth is the lidar weight and cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim_20201009_124951-40963960.pth. is the camera weight. However, how should I put both of them together in the inference code though?

Much appreciate!