BEVFormer: a Cutting-edge Baseline for Camera-based Detection

注：本代码库在BEVFormer算法所提供的代码的基础上新增了BEV分割代码(Note: This code library adds BEV segmentation code based on the code provided by the BEVFormer algorithm)

https://user-images.githubusercontent.com/72087870/232678436-b0b51b1b-fe09-4ed1-aadd-be2589360276.mp4

Abstract

In this work, the authors present a new framework termed BEVFormer, which learns unified BEV representations with spatiotemporal transformers to support multiple autonomous driving perception tasks. In a nutshell, BEVFormer exploits both spatial and temporal information by interacting with spatial and temporal space through predefined grid-shaped BEV queries. To aggregate spatial information, the authors design a spatial cross-attention that each BEV query extracts the spatial features from the regions of interest across camera views. For temporal information, the authors propose a temporal self-attention to recurrently fuse the history BEV information. The proposed approach achieves the new state-of-the-art 56.9\% in terms of NDS metric on the nuScenes test set, which is 9.0 points higher than previous best arts and on par with the performance of LiDAR-based baselines.

Methods

Getting Started

• Installation
• Prepare Dataset
• Run and Eval

Model Zoo

origin

• [x] 3D Detection checkpoints
• [x] 3D Detection code
• [x] Initialization

New

• [x] BEV Segmentation checkpoints
• [x] BEV Segmentation code
• [x] multi-task visualization
• [x] weight

改进(Improve)

map分支：仅训练分割分支，目前只支持单卡训练(map branch: only training split branch, currently only supports single card training)

双分支：实现了检测与分割的联合训练，并支持并行(Double branch: realize the joint training of detection and segmentation, and support parallelism)

det：

seg：

Acknowledgement

本项目参考了BEVerse, open-mmlab, BEVDet, HDMapNet等代码库，在这里表示感谢！(This project refers to code libraries such as BEVerse, open-mmlab, BEVDet, HDMapNet, etc. Thank you here!)