News: I've released my new work on MOT. This tracker is named as Deep Motion Modeling Network (DMMN) which can simultaneously perform detection, feature modeling, association, and even prediction. Besides, this work is based on the framework of SSD, which can be extended to the framework of Faster RCNN, YOLO v3/4.

DAN (Deep Affinity Network)

Table Of Content

• DAN (Deep Affinity Network)
  • Table Of Content
  • Purpose
  • Deep Affinity Network (DAN)
    • The input & output of network
      • Training phase
      • Testing phase
  • Task
    • Current Task
    • History Task
  • Requirement
  • Dataset
  • Train & Test On MOT17
    • Download dataset
    • Test
    • Train
  • Citation
  • Acknowledge
  • License
  • Cool Examples

Purpose

DAN is an end-to-end deep learning network during train phase, whose purpose is to extract the predetected objects' feature and performs pairing permutations of those features in any two frames to infer object affinities. Besides, DAN also accounts for multiple objects appearing and disappearing between video frames.

Note: The repository was built with the name "SST". For the brevity and easy understanding, we change the repository name by "DAN".

Deep Affinity Network (DAN)

The network can be divided into two parts: feature extractor and affinity extractor. The feature extractor extracts each detected objects' features. The affinity extractor leverage those features to compute object data association matrix.

Deep tracking with DAN deployment. The DAN extracts the feature of each detected object provided by the Detector, and also estimate the affinity matrices.

The input & output of network

Training phase

Testing phase

Task

Current Task

History Task

Requirement

The requirement as follows:

Before going on, we recommend minconda. After you install miniconda, then create a new environment named DAN, then run the following script:

conda create -n DAN python=3.5
source activate DAN

Run the following script to install the required python packages:

If you're in China, you'd better run the following script in order to speed up the downloading.

pip install pip -U
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

cd SST
pip install -r requirement.txt

SST is the path of our repository folder.

Dataset

Our method can be evaluated on MOT17, MOT15 and UA-DETRAC.

MOT15 and MOT17 is for pedestrian tracking.

UA-DETRAC focuses is for vehicle tracking.

Train & Test On MOT17

For the simplity, we only introduce how to train and evaluate on MOT17

Download dataset

1. Download the mot 17 dataset 5.5 GB and development kit 0.5 MB.

2. Unzip this the dataset. Here is my file structure.

   MOT17
   ├── test
   └── train

Test

1. Download the weigths from Dropbox or BaiYunPan to the SST/weights folder

2. Modify SST/config/config.py as follows:

   # You need to modify line 8, 72, 73 and 74.
   8    current_select_configure = 'init_test_mot17' # need use 'init_test_mot17'
   ...  ...
   70   def init_test_mot17():
   71        config['resume'] = './weights/sst300_0712_83000.pth'
   72        config['mot_root'] = 'replace by your dataset folder' 
   73         config['save_folder'] = 'replace by your save folder'
   74        config['log_folder'] = 'replace by your log folder'
   75        config['batch_size'] = 1
   76        config['write_file'] = True
   77        config['tensorboard'] = True
   78        config['save_combine'] = False
   79        config['type'] = 'test' # or 'train'

3. run test_mot17.py

   cd <SST>
   python test_mot17.py

   The result is shown as follows

   

   The title of each detected boxes represents (track id, detection id)

Train

1. Download the vgg weights from Dropbox or BaiYunPan to the weights folder

2. Modify SST/config/config.py as follows:

   # you need to modify line 8, 87, 89, 90 and 91.
   8    current_select_configure = 'init_train_mot17' # need use 'init_train_mot17'
   ...  ...
   85   def init_train_mot17():
   86       config['epoch_size'] = 664
   87       config['mot_root'] = 'replace by your mot17 dataset folder'
   88       config['base_net_folder'] = './weights/vgg16_reducedfc.pth'
   89       config['log_folder'] = 'replace by your log folder'
   90       config['save_folder'] = 'replace by your save folder'
   91       config['save_images_folder'] = 'replace by your image save folder'
   92       config['type'] = 'train'
   93       config['resume'] = None # None means training from sketch.
   94       config['detector'] = 'DPM'
   95       config['start_iter'] = 0
   96       config['iteration_epoch_num'] = 120
   97       config['iterations'] = config['start_iter'] + config['epoch_size'] *     config['iteration_epoch_num'] + 50
   98       config['batch_size'] = 4
   99       config['learning_rate'] = 1e-2
   100      config['learning_rate_decay_by_epoch'] = (50, 80, 100, 110)
   101      config['save_weight_every_epoch_num'] = 5
   102      config['min_gap_frame'] = 0
   103      config['max_gap_frame'] = 30
   104      config['false_constant'] = 10
   105      config['num_workers'] = 16
   106      config['cuda'] = True
   107      config['max_object'] = 80
   108      config['min_visibility'] = 0.3

3. Run train_mot17.py

   cd <SST>
   python train_mot17.py

Citation

If you use this source code or part of the source code. It is necessary to cite the following paper:

Sun. S., Akhtar, N., Song, H., Mian A., & Shah M. (2018). Deep Affinity Network for Multiple Object Tracking, Retrieved from https://arxiv.org/abs/1810.11780

Acknowledge

This code is based on ssd.pytorch

License

The methods provided on this page are published under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License . This means that you must attribute the work in the manner specified by the authors, you may not use this work for commercial purposes and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same license. If you are interested in commercial usage you can contact us for further options.

Cool Examples