search

JonathonLuiten / TrackEval

HOTA (and other) evaluation metrics for Multi-Object Tracking (MOT).
MIT License
965 stars 239 forks source link

TrackEvalException: Evaluation is only valid for pedestrian class #83

Closed alemelis closed 2 years ago

alemelis commented 2 years ago

Hello, I'm trying to run the MOT challenge script on a custom dataset. I think I've got the folder structure right (including all the .ini files) and my gt files look like

1,1,304,938,79,140,-1,0,-1
1,2,558,264,22,50,-1,0,-1
...

I run the script as

python3 scripts/run_mot_challenge.py --GT_FOLDER /Users/a/TrackEval/data/gt/mydata --TRACKERS_FOLDER /Users/a/TrackEval/data/trackers/mydata --BENCHMARK tracker-bm --SPLIT_TO_EVAL test --TRACKERS_TO_EVAL mytracker --DO_PREPROC False

All the tracking data is correctly found and the script outputs

Evaluating 1 tracker(s) on 6 sequence(s) for 1 class(es) on MotChallenge2DBox dataset using the following metrics: HOTA, CLEAR, Identity, Count

however it stops with the following error

trackeval.utils.TrackEvalException: Evaluation is only valid for pedestrian class. Non pedestrian class (6) found in sequence first_video at timestep 2.

In the ground truth there's no mention of class (the MOT format doesn't contain that), where does the script get that class 6? Also, I'm not sure what the CLASSES_TO_EVAL argument is supposed to do (by default it is set to pedestrian).

Setting --CLASSES_TO_EVAL 6 raises

trackeval.utils.TrackEvalException: Attempted to evaluate an invalid class. Only pedestrian class is valid.
alemelis commented 2 years ago

actually, the 6 is in my tracker output, it's the object-class I was under the impression that the last 3 numbers would be ignored...

I guess the question now is: provided I change my gt to include class information, how would I run the benchmark on a multi-class setup?

mystarry commented 1 year ago

trackeval.utils.TrackEvalException: Attempted to evaluate an invalid class. Only pedestrian class is valid. May I ask how you resolved the above issues？

mystarry commented 1 year ago

I tried to replace the pedestrian in the evaluation file with my category name, but it didn't work.

mesllo-bc commented 1 year ago

Any updates on this please? I also have come across this issue.

Suyufeng123 commented 3 months ago

Any updates on this please? I also have come across this issue.