search

shenh10 / mot_evaluation

A python implementation of Multiple Object Tracking(MOT) evaluation toolkit(2D)
52 stars 21 forks source link

Why the MOTA and IDF1 are not 100% when I input the ground truth as the tracking results? #3

Closed Reeshark closed 5 years ago

Reeshark commented 5 years ago

Why the MOTA and IDF1 are not 100% when I input the ground truth as the tracking results? For example your data MOT16-13, which MOTA is only 31.8 and IDF1 is only 74.6?

shenh10 commented 5 years ago

Hi, thanks for trying out this code. It is because MOT16 groundtruth includes some extra non-people classes like walls, etc. Thus there would be 'FP' in evaluation results. For example, use the gt.txt as detection results of MOT16-02-DPM. Then there would be 3402 fps. Which is expected and reasonable.

[GT PREPROCESSING]: Removing non-people classes, remaining 26601/30003 boxes
[TRACK PREPROCESSING]: remove distractors and low visibility boxes, remaining 21983/30003 computed boxes
('Distractors:', array([  1.,   5.,   6.,  11.,  12.,  13.,  24.,  25.,  30.,  45.,  46.,
        47.,  65.,  66.]))
[GT PREPROCESSING]: Removing distractor boxes, remaining 18581/26601 computed boxes
('\n', '******************************', 'MOT17-02 Evaluation', '******************************')
IDF1  IDP  IDR  | Rcll  Prcn  FAR   | GT   MT   PT   ML   | FP     FN     IDs   FM    | MOTA  MOTP  MOTAL
 91.6 84.5 100.0 | 100.0  84.5  5.67 |   62   62    0    0 |   3402      0     0     0 |  81.7 100.0  81.7
('\n', '******************************', 'Summary Evaluation', '******************************')
IDF1  IDP  IDR  | Rcll  Prcn  FAR   | GT   MT   PT   ML   | FP     FN     IDs   FM    | MOTA  MOTP  MOTAL
 91.6 84.5 100.0 | 100.0  84.5  5.67 |   62   62    0    0 |   3402      0     0     0 |  81.7 100.0  81.7
Reeshark commented 5 years ago

Thanks for reply, I have realized this issue. Your explaination is same as my conclusion.

Reeshark commented 5 years ago

I have read your paper "Tracklet Association Tracker: An End-to-End Learning-based Association Approach for Multi-Object Tracking"

shenh10 commented 5 years ago

I have read your paper "Tracklet Association Tracker: An End-to-End Learning-based Association Approach for Multi-Object Tracking"

Thank you for your attention. I hope the paper and this code can be helpful for you. FYI, I updated the code to python3 today and polish the coding styles by linter.