Update on 2021.2.4: New data, trained models, results, and categories have been released! Old version data is not available now.

Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimation (CVPR 2020 and T-PAMI 2021)

This project contains the implementation of our paper arxiv.

Authors: Jiaming Sun, Linghao Chen, Yiming Xie, Siyu Zhang, Qinhong Jiang, Xiaowei Zhou, Hujun Bao.

Requirements

Ubuntu 16.04+
Python 3.7+
8 Nvidia GPU with mem >= 12G (recommended, see Notes for details.)
GCC >= 4.9
PyTorch 1.2.0

Install

# Install webp support
sudo apt install libwebp-dev
# Clone repo
git clone https://github.com/zju3dv/disprcnn.git
cd disprcnn
# Install conda environment
conda env create -f environment.yaml
conda activate disprcnn
# Install Disp R-CNN
sh build_and_install.sh

Training and evaluation

See TRAIN_VAL.md

Sample results

Citation

If you find this code useful for your research, please use the following BibTeX entry.

@inproceedings{sun2020disprcnn,
  title={Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimation},
  author={Sun, Jiaming and Chen, Linghao and Xie, Yiming and Zhang, Siyu and Jiang, Qinhong and Zhou, Xiaowei and Bao, Hujun},
  booktitle={CVPR},
  year={2020}
}
@article{chen2021shape,
  title={Shape prior guided instance disparity estimation for 3d object detection},
  author={Chen, Linghao and Sun, Jiaming and Xie, Yiming and Zhang, Siyu and Shuai, Qing and Jiang, Qinhong and Zhang, Guofeng and Bao, Hujun and Zhou, Xiaowei},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  volume={44},
  number={9},
  pages={5529--5540},
  year={2021},
  publisher={IEEE}
}

Acknowledgment

This repo is built based on the Mask R-CNN implementation from maskrcnn-benchmark, and we also use the pretrained Stereo R-CNN weight from here for initialization. The system architure figure is created with Blender, feel free to reuse our project file!

Copyright

This work is affiliated with ZJU-SenseTime Joint Lab of 3D Vision, and its intellectual property belongs to SenseTime Group Ltd.

Copyright SenseTime. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

zju3dv / disprcnn

readme