hailanyi / 3D-Multi-Object-Tracker

A project for 3D multi-object tracking
Apache License 2.0
302 stars 55 forks source link
3d-object-tracking kitti multi-object-tracking

3D Multi-Object Tracker

This project is developed for tracking multiple objects in 3D scene. The visualization code is from here. This is a re-produced and simplified version of paper. We have made some changes to support more detectors and adapt the HOTA metric. We also provided a simple guide to submit results to the KITTI test set.

Features

Kitti Results

Car/Pedestrian tracking results on the Kitti tracking val seq [1,6,8,10,12,13,14,15,16,18,19] using second-iou, point-rcnn and pv-rcnn detections. We also followed the HOTA metric, and tuned the parameters by first considering the HOTA performance.

detector online near online (latency=2s) global
point-rcnn 76.653 77.576 78.853
second-iou 77.29/48.011 78.17/50.362 78.799/51.20
pv-rcnn 78.289/49.662 79.48/50.824 80.075/51.753

Online: CA-based KF + greedy matching. Near online: online + rescoring tracks in a temporal window. Global: online + rescoring tracks globally.

Detector online near online (latency=2s) global detections (training & testing set)
CasA 82.59 83.24 83.34 detections download link(40M)
Detector online near online (latency=2s) global detections (training & testing set)
VirConv 85.40 85.98 86.17 detections download link(8M)

Waymo Results

Vehicle/Pedestrian/Cyclist online tracking results, evaluated by MOTA/L2 metrics of Waymo benchmark.

detector set Vehicle_L2 Pedestrian_L2 Cyclist_L2 ALL_NS_L2
CasA val set 59.30 62.78 61.72 61.27
CasA test set 63.66 64.79 59.34 62.60

Prepare data

You can download the Kitti tracking pose data from here, and you can download the point-rcnn, second-iou and pv-rcnn detections from here. You can download the CasA detections(including training & testing set) from here.

To run this code, you should organize Kitti tracking dataset as below:

# Kitti Tracking Dataset       
└── kitti_tracking
       ├── testing 
       |      ├──calib
       |      |    ├──0000.txt
       |      |    ├──....txt
       |      |    └──0028.txt
       |      ├──image_02
       |      |    ├──0000
       |      |    ├──....
       |      |    └──0028
       |      ├──pose
       |      |    ├──0000
       |      |    |    └──pose.txt
       |      |    ├──....
       |      |    └──0028
       |      |         └──pose.txt
       |      ├──label_02
       |      |    ├──0000.txt
       |      |    ├──....txt
       |      |    └──0028.txt
       |      └──velodyne
       |           ├──0000
       |           ├──....
       |           └──0028      
       └── training # the structure is same as testing set
              ├──calib
              ├──image_02
              ├──pose
              ├──label_02
              └──velodyne 

Detections

└── point-rcnn
       ├── training
       |      ├──0000
       |      |    ├──000001.txt
       |      |    ├──....txt
       |      |    └──000153.txt
       |      ├──...
       |      └──0020
       └──testing 

Requirements

python3
numpy
opencv
yaml

Quick start

Notes

The evaluation codes are copied from Kitti.