search

cheind / py-motmetrics

:bar_chart: Benchmark multiple object trackers (MOT) in Python
MIT License
1.37k stars 258 forks source link

unexpected result when inputing ground truth as test set #66

Open trincty opened 4 years ago

trincty commented 4 years ago

Hi, I'm using the CLI and I'm getting confused with what it is doing. I tried using the same dataset as ground truth data and test data. In my understanding, the test data is my hypothesis of the tracker. So, if I input the same dataset, there should be no false positive. But I get this result. I'm using the MOT16-02 form motchallenge. IDF1 IDP IDR Rcll Prcn GT MT PT ML FP FN IDs FM MOTA MOTP MOT16-02 75.8% 61.1% 100.0% 100.0% 61.1% 54 54 0 0 11360 0 0 0 36.3% 0.000 OVERALL 75.8% 61.1% 100.0% 100.0% 61.1% 54 54 0 0 11360 0 0 0 36.3% 0.000

cheind commented 4 years ago

this might be caused by duplicate ids in single frames as reported in #60.

SamSamhuns commented 4 years ago

In https://github.com/cheind/py-motmetrics/issues/48, when only observations with label ID=1 (Pedestrian) in the MOT16-02 challenge is used, inputting ground truth as the test set gives expected results.

Also, @cheind, I checked and there are no duplicate ids in single frames for the MOT16-02 data. The issue might have something to do with the label.

Screen Shot 2020-07-12 at 5 45 57 PM
cheind commented 4 years ago

Interesting. I hope to find some time to inspect soon.

cheind commented 4 years ago

Might be related to #115