Towards Robust Human Trajectory Prediction in Raw Videos

The repository contains the code and data for "Towards Robust Human Trajectory Prediction in Raw Videos" IROS 2021.

Dataset

SDD

Download the Stanford Drone Dataset (SDD). The ~69G zip file includes the raw videos and annotations.

• Our refined annotations of SDD 📢

  • We found many outlier bounding boxes in the original annotations via visualization. To better evaluate the tracking, we refined the annotations by removing the outliers.
  • Our refined annotation files have been uploaded to Google Drive.

WILDTRACK

Download the WILDTRACK Seven-Camera HD Dataset.

• Our extended annotations of WILDTRACK 📢

  • The original dataset annotated the first 400 frames of each of the seven videos at 2 fps. To better facilitate the behavior learning, we manually annotated the first 900 frames (refined 1\~400 and new 401\~900) for each video with the annotation tool.
  • The extended annotation files have been uploaded to Google Drive, including the original WILDTRACK format and the MOT format.
  • Our annotations can be used for larger-scale evaluation on WILDTRACK for single-view/multi-view pedestrian detection, tracking, and trajectory prediction.

Installation

The project was developed based on Python 3.7.9 and PyTorch 1.1.0. The environment can be set up via the environment.yml file:

conda env create -f environment.yml
conda activate retracking

Experiments

As an example, we show how to run the experiments on the SDD dataset.

1. Train a prediction model

   cd retracking-by-prediction
   python train.py

2. Detection, Tracking, Prediction, and Re-tracking

   sh run_test.sh

3. Evaluation

   We evaluate the tracking performance via py-motmetrics. Note: we use L2 distance in meter instead of box IoU as the association metric.

Citation

If you use the code or data in your research, please cite the paper:

@inproceedings{Yu2021-Retracking,
  author    = {Rui Yu and Zihan Zhou},
  title     = {Towards Robust Human Trajectory Prediction in Raw Videos},
  booktitle = {{IEEE/RSJ} International Conference on Intelligent Robots and Systems, {IROS}},
  year      = {2021}
}

License

The project is released under the MIT License. The SORT tracking code should follow its own license (GPL-3.0).