BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection

Lei Yang · Kaicheng Yu · Tao Tang · Jun Li · Kun Yuan · Li Wang · Xinyu Zhang · Peng Chen

CVPR 2023

BEVHeight is a new vision-based 3D object detector specially designed for roadside scenario. BEVHeight surpasses BEVDepth base- line by a margin of 4.85% and 4.43% on DAIR-V2X-I and Rope3D benchmarks under the traditional clean settings, and by 26.88% on robust settings where external camera parameters changes. We hope our work can shed light on studying more effective feature representation on roadside perception.

News

• [2024/05] BEVHeight is integrated into NVIDIA DeepStream-3D for sensor fusion.
• [2023/03/15] Both arXiv and codebase are released!
• [2023/02/27] BEVHeight got accepted to CVPR 2023!

Incoming

• [ ] Release the pretrained models
• [ ] Support train and test on a custom dataset

Getting Started

• Installation
• Prepare Dataset

Train BEVHeight with 8 GPUs

python [EXP_PATH] --amp_backend native -b 8 --gpus 8

Eval BEVHeight with 8 GPUs

python [EXP_PATH] --ckpt_path [CKPT_PATH] -e -b 8 --gpus 8

Experimental Results

• DAIR-V2X-I Dataset

  Method	Config File	Range	Car			Pedestrain			Cyclist			model pth
  			3D@0.5			3D@0.25			3D@0.25
  			Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
  BEVHeight	R50_102	[0, 102.4]	77.48	65.46	65.53	26.86	25.53	25.66	51.18	52.43	53.07	model
  	R50_140	[0, 140.8]	80.80	75.23	75.31	28.13	26.73	26.88	49.63	52.27	52.98	model
  	R101_102	[0, 102.4]	78.06	65.94	65.99	40.45	38.70	38.82	57.61	59.90	60.39	model
  	R101_140	[0, 140.8]	81.80	76.19	76.26	38.79	37.94	38.26	58.22	60.49	61.03	model

  Rope3D Dataset

  hom_train.pkl

  hom_val.pkl

  Method	Config File	Range	Car | 3D@0.5			Big Vehicle | 3D@0.5			Car | 3D@0.7			Big Vehicle | 3D@0.7			model pth
  			Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
  BEVHeight	R50_102	[0, 102.4]	83.49	72.46	70.17	50.73	47.81	47.80	48.12	42.45	42.34	24.58	26.25	26.28	model
  	R50_140	[0, 140.8]	85.46	79.15	79.06	64.38	65.75	65.77	46.39	42.85	42.71	27.21	33.99	34.03	model

  Acknowledgment

  This project is not possible without the following codebases.

  • BEVDepth
  • DAIR-V2X
  • pypcd

  Citation

  If you use BEVHeight in your research, please cite our work by using the following BibTeX entry:

  @inproceedings{yang2023bevheight,
      title={BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection},
      author={Yang, Lei and Yu, Kaicheng and Tang, Tao and Li, Jun and Yuan, Kun and Wang, Li and Zhang, Xinyu and Chen, Peng},
      booktitle={IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
      month = mar,
      year={2023}
  }