search

JonathonLuiten / TrackEval

HOTA (and other) evaluation metrics for Multi-Object Tracking (MOT).
MIT License
963 stars 239 forks source link

GT not loaded for ai_city data in MOTChallenge format #25

Closed JonathonLuiten closed 3 years ago

JonathonLuiten commented 3 years ago

Hi,

Thank you for your great work with the HOTA metrics. I've a related issue but this is not about the difference between 9 and 10 values per line (even though I tried to remove one to see if it helps, but it does not).

I tried your script with a GT file from the AICity challenge 2021 and compared it with a pred file they provide, both following the MOT format (attached to this message), but for some unknown reason, the script does not read the GT file, same as if it was empty:

Count: DeepSORT-SSD512-pedestrian  Dets      GT_Dets   IDs       GT_IDs
s01-c003                           14151     0         219       0
COMBINED                           14151     0         219       0

I tried to change some value, indexed by obj_id instead of frame_id, remove one '-1' at the end of each line, checked the encoding and also checked the entire hierarchy of 'trackers' and 'gt' folders.

Finally, I copied a GT file from a MOT sequence (MOT16-train\MOT16-02) and pasted it as it was my own GT file, even if it doesn't match at all... and it reads the GT file correctly:

Count: DeepSORT-SSD512-pedestrian  Dets      GT_Dets   IDs       GT_IDs
s01-c003                           14151     17833     219       54
COMBINED                           14151     17833     219       54

Here's the original GT file of the sequence I want to test: gt.txt

Here's one of the prediction file: s01-c003.txt

Thank you in advance for your help.

Originally posted by @JonathanSamelson in https://github.com/JonathonLuiten/TrackEval/issues/23#issuecomment-814948323

JonathonLuiten commented 3 years ago

@JonathanSamelson

I just made this a new issue, because it's a different problem to the other issue.

I have just made a push that solves this.

Note that the problem was that you were using gt data in MOT15 format (with no classes), using the default code which assumes data in MOT16/17/20 format, for which there are classes.

Before the code just didn't read anything (that was a bug, thanks for pointing it out), now it raises and error that directs you how to fix this (by using the flag option --DO_PREPROC False)

This fixes your error, so closing this now.

Cheers, Jono

JonathanSamelson commented 3 years ago

Indeed, it's working great now! Thank you!