search

NVIDIA-AI-IOT / Lidar_AI_Solution

A project demonstrating Lidar related AI solutions, including three GPU accelerated Lidar/camera DL networks (PointPillars, CenterPoint, BEVFusion) and the related libs (cuPCL, 3D SparseConvolution, YUV2RGB, cuOSD,).
Other
1.27k stars 224 forks source link

BEVFusion: how to get/train the pretrained resnet50xxx.pth and swin_xxx.pth #117

Closed bigsheep2018 closed 1 year ago

bigsheep2018 commented 1 year ago

Hello @hopef I found a issue mentioned the problem on official bevfusion repo. Link: https://github.com/mit-han-lab/bevfusion/issues/176 It shows a quite complicated way for getting the pre-trained swin_xxx.pth. I.e., we need to train a CascadeRCNN on Nuscenes 2D detection task following a pretrained Swin-T on COCO 2D detection task. Well, as we need to train bevfusion from scratch on customized dataset (2D det pretrain --> 3D det bevfusion), there are 2 questions need to be checked:

  1. Is there an easier way to get the pretrained swin_xxx.pth and resnet50xxx.pth? Or maybe is it possible to release the training config/code of these pretrained .pths?
  2. For using Resnet50xxx.pth, are you guys just skipping the pre-training procedures on COCO/nuScenes 2D Det task and using the default pytorch pretrained model directly?
hopef commented 1 year ago

https://github.com/open-mmlab/mmdetection3d/tree/1.0/configs/nuimages

bigsheep2018 commented 1 year ago

https://github.com/open-mmlab/mmdetection3d/tree/1.0/configs/nuimages

Thanks for the reply @hopef . I would try Cascade MaskRCNN and HTC's backbones, then get back to this issue.

bigsheep2018 commented 1 year ago

https://github.com/open-mmlab/mmdetection3d/tree/1.0/configs/nuimages

Hello @hopef , Using HTC's pretrained resnet50 backbone, we got MAP 67.98 and NDS 71.03 on Nuscenes Validation set. Closing the issue.