3D Small Object Detection with Dynamic Spatial Pruning

Introduction

This repo contains PyTorch implementation for paper 3D Small Object Detection with Dynamic Spatial Pruning based on MMDetection3D. Look here for 中文解读.

3D Small Object Detection with Dynamic Spatial Pruning
Xiuwei Xu, Zhihao Sun\, Ziwei Wang, Hongmin Liu, Jie Zhou, Jiwen Lu

teaser

News

[2024/7/01]: DSPDet3D is accepted to ECCV 2024!
[2023/6/04]: We transfer DSPDet3D to extremely large scenes and show great performance! Demo will be released in our project page.
[2023/5/01]: Code release.

Method

Overall pipeline of DSPDet3D:

overview

Getting Started

For data preparation and environment setup:

For training and evaluation:

Train and Eval

Demo

This is a tutorial on how to use class-agnostic DSPDet3D on custom data and visualize the results. Please download checkpoint from HERE and move it to demo folder. We provide two demo scenes from ScanNet and Matterport3D. You can download (ScanNet, Matterport3D) and also put them into demo folder. Then run the following command for detection and visualization.

Dataset	Scannet	Matterport3D
Command	`bash demo/demo.sh demo/scannet.ply demo/config_room.py`	`bash demo/demo.sh demo/mp3d.ply demo/config_building.py`
Result

You can also try DSPDet3D on your own data in ply format. Run

bash demo/demo.sh /path/to/your/ply demo/config_{}.py

We use different hyperparamters of 3D NMS for different scales of scenes. For room-size scenes, use config_room.py. For building-level scenes, use config_building.py. You can also adjust the prune_threshold in the config file to tradeoff between accuracy and efficiency.

Main Results

We provide the checkpoints for quick reproduction of the results reported in the paper. The pruning threshold can be adjusted freely to tradeoff between accuracy and efficiency without any finetuning. Benchmark	mAP@0.25	mAP@0.5	Downloads
ScanNet-md40	65.04	54.35	model
TO-SCENE-down	66.12	58.55	model

Comparison with state-of-the-art methods on TO-SCENE dataset:

drawing

Visualization results on ScanNet:

vis

Visualization results on Matterport3D:

vis2

Acknowledgement

We thank a lot for the flexible codebase of FCAF3D and valuable datasets provided by ScanNet and TO-SCENE.

Bibtex

If this work is helpful for your research, please consider citing the following BibTeX entry.

@article{xu2023dsp, 
      title={3D Small Object Detection with Dynamic Spatial Pruning}, 
      author={Xiuwei Xu and Zhihao Sun and Ziwei Wang and Hongmin Liu and Jie Zhou and Jiwen Lu},
      journal={arXiv preprint arXiv:2305.03716},
      year={2023}
}

xuxw98 / DSPDet3D

readme