search

MCG-NJU / MOTIP

Multiple Object Tracking as ID Prediction
https://arxiv.org/abs/2403.16848
Apache License 2.0
76 stars 7 forks source link
computer multi-object-tracking multiple-object-tracking tracking

readme

Multiple Object Tracking as ID Prediction

This is the official PyTorch implementation of our paper:

Multiple Object Tracking as ID Prediction
:mortar_board: Ruopeng Gao, Yijun Zhang, Limin Wang
:e-mail: Primary contact: ruopenggao@gmail.com

PWC
PWC
PWC

:mag: Overview

TL; DR. MOTIP proposes a new perspective to regard the multi-object tracking task as an ID prediction problem. It directly predicts the ID labels for each object in the tracking process, which is more straightforward and effective.

Overview

Abstract. In Multiple Object Tracking (MOT), tracking-by-detection methods have stood the test for a long time, which split the process into two parts according to the definition: object detection and association. They leverage robust single-frame detectors and treat object association as a post-processing step through hand-crafted heuristic algorithms and surrogate tasks. However, the nature of heuristic techniques prevents end-to-end exploitation of training data, leading to increasingly cumbersome and challenging manual modification while facing complicated or novel scenarios. In this paper, we regard this object association task as an End-to-End in-context ID prediction problem and propose a streamlined baseline called MOTIP. Specifically, we form the target embeddings into historical trajectory information while considering the corresponding IDs as in-context prompts, then directly predict the ID labels for the objects in the current frame. Thanks to this end-to-end process, MOTIP can learn tracking capabilities straight from training data, freeing itself from burdensome hand-crafted algorithms. Without bells and whistles, our method achieves impressive state-of-the-art performance in complex scenarios like DanceTrack and SportsMOT, and it performs competitively with other transformer-based methods on MOT17. We believe that MOTIP demonstrates remarkable potential and can serve as a starting point for future research.

:fire: News

:chart_with_upwards_trend: ​Main Results

:dancer: ​DanceTrack

Method Training Data HOTA DetA AssA MOTA IDF1 URLs
MOTIP DT 67.5 79.4 57.6 90.3 72.2 model, config, log
MOTIPDAB DT 70.0 80.8 60.8 91.0 75.1 model, config, log
MOTIP DT + CH 71.4 81.3 62.8 91.6 76.3 model, config, log
MOTIP DT* + CH 73.7 82.6 65.9 92.7 78.4 model, config, log
NOTE
  1. MOTIP is built upon original Deformable DETR, while MOTIPDAB is based on DAB-Deformable DETR.
  2. DT and CH are the abbreviations of DanceTrack and CrowdHuman respectively.
  3. DT* denotes we utilize both the training and validation set of DanceTrack for training.

:basketball: ​SportsMOT

Method Training Data HOTA DetA AssA MOTA IDF1 URLs
MOTIP Sports 71.9 83.4 62.0 92.9 75.0 model, config, log
MOTIP Sports* 75.2 86.5 65.4 96.1 78.2 model, config*, log
NOTE
  1. Sports is the abbreviation of SportsMOT.
  2. Sports* denotes we utilize both the training and validation set of SportsMOT for training.
  3. config* represents the configuration that can be used for inference. The corresponding training config file has not been uploaded yet.

:walking: ​MOT17

Method Training Data HOTA DetA AssA MOTA IDF1 URLs
MOTIP MOT17 + CH 59.2 62.0 56.9 75.5 71.2 model, config, log
NOTE
  1. CH is the abbreviation of CrowdHuman.

:dash: Quick Start

:bouquet: Acknowledgements

This project is built upon Deformable DETR, MOTR, DAB-Deformable DETR, TrackEval. Thanks to the contributors of these great codebases.

:pencil2: Citation

If you think this project is helpful, please feel free to leave a :star: and cite our paper:

@article{MOTIP,
  title={Multiple Object Tracking as ID Prediction},
  author={Gao, Ruopeng and Zhang, Yijun and Wang, Limin},
  journal={arXiv preprint arXiv:2403.16848},
  year={2024}
}