search

JonathonLuiten / TrackEval

HOTA (and other) evaluation metrics for Multi-Object Tracking (MOT).
MIT License
968 stars 241 forks source link

Regarding track mAP on TAO dataset #48

Open Zhongdao opened 3 years ago

Zhongdao commented 3 years ago

Hi,

Thanks for the excellent work! I found HOTA had become one of the most popular metrics for MOT nowadays, everyone in the tracking community is using it.

I am recently trying to test trackers on the TAO dataset. When I run TrackEval and the official TAO evaluation script, I find their outputs of track mAP are inconsistent. I am not sure if this is due to my mistake, or if the two implementations of track mAP are actually different? Should I expect the two results to match perfectly?

By the way, I used the off-the-shelf results from AOA, and test on the val split. The output of the official eval code is around 29 mAP, while the output of this codebase is around 16 mAP.

Zhongdao commented 3 years ago

Another minor thing: The result JSON mentioned above contain duplicated track ids in single frames, and the TrackEval tool would not accept such results. So I removed those duplicated boxes. I am not sure if this is the reason.

xuaixi commented 3 years ago

image

when I use the track_map, there is the problem I met. I want to ask you how to deal with it? Thank you

Zhongdao commented 3 years ago

@xuaixi I guess you are using the wrong format in the MOT results files (those .txt files). By the way, if you are working on multi-class MOT, we just released a nice baseline (better than YOLOv5+DeepSORT). See here: YOLOX+UniTrack. Hope these help :)