Is Pseudo-Lidar needed for Monocular 3D Object detection?

[AtsukiOsanai]

• ICCV2021
• arXiv
• github

[AtsukiOsanai]

Introduction

• Ordinal pipeline
  • two stages
  • 1st: mono --> depth estimation
  • 2nd: pseudo lidar --> 3d object detection
  • The performance depends on the accuracy of depth estimation
• Proposed
  • single stage 3d od
  • monocular depth estimation for pre-training
• The difference
  • Most of 3d detectors require additional data (3D CAD, instance seg)
  • Some methods regress 3d bbox directly by adding heads
  • This method focus on the self-supervised depth pre-training to get good depth representation, which leads to robust 3d od

[AtsukiOsanai]

Methods

• Arch based on FCOS
• camera-parameter aware depth estimation
• Loss consists of L_2d (detection on 2d), L_3d(3d box regression), and self-supervised loss for 3d box confidence

[AtsukiOsanai]

Depth pre-training

• depth supervised training by using valid lidar measurement
• 3d detection head and 3d dense depth prediction head differ only in the last 3x3 conv, which leads to effective feature transfer

[AtsukiOsanai]

Pseudo-Lidar

• PackNet for dense depth estimation
• Frustum pointnets like detector
• backbone with ResNet18+SE module

[AtsukiOsanai]

Results

• Among monocular 3d detection models, the proposed method achieves the state-of-the-art
  • Compared to Lidar-based 3d detection, the performance gap is still large. Please see here to get the score for them.

[AtsukiOsanai]

Analysis

• depth pre-training on DDAD15M is more important than COCO pre-training
• depth pre-training is more important than detection pre-training on nuscenes dataset
• The 3d detection accuracy scales against the amount of depth pre-training data up to 15M
• PL requires depth fine-tuning(Eigen clean) on the target domain. The end-to-end approaches can be worked for a new domain.
• PL achieves excellent performance on KITTI 3d val, yet it has not been reflected on the test set. It shows the difficulty of generalization on the PL-based method.

[AtsukiOsanai]