vukasin-stanojevic / BoostTrack

MIT License
84 stars 12 forks source link

Reproduce paper result #6

Closed trongldwork closed 3 months ago

trongldwork commented 3 months ago

I clone your repo and do setting like instruction, but i cant reproduce the result in your paper, with --no_reid and full method you provide in paper, my result is image

vukasin-stanojevic commented 3 months ago

Hello. The original results from the paper are obtained on the machine with very unusual python - conda setup. There was a lot of dependency conflicts which I somehow managed to resolve. The result was that somehow it used some packages from the base interpreter and some from the conda environment.

After submitting the paper, I tried to setup repository from the start and manage to do it with different package versions (resulting in the simple setup described in the readme file). The results obtained with this new version are slightly different (usually the difference is only on the second decimal and "unbiased" - better for some metrics and worse for the other).

Based on the numbers you present, I assume you tried it on MOT20. Below I list the results from the paper:

MOT17 validation set: Method HOTA MOTA IDF1 IDSW
BoostTrack 71.63 80.692 83.959 97
BoostTrack+ 72.411 80.67 85.385 81
MOT20 validation set: Method HOTA MOTA IDF1 IDSW
BoostTrack 61.846 77.442 77.584 882
BoostTrack+ 62.68 77.731 79.088 781

The results I get with different package versions:

MOT17 validation set: Method HOTA MOTA IDF1 IDSW
BoostTrack 71.532 80.7 83.875 104
BoostTrack+ 72.407 80.941 85.091 83
MOT20 validation set: Method HOTA MOTA IDF1 IDSW
BoostTrack 61.742 77.464 77.454 898
BoostTrack+ 62.691 77.717 79.081 787

The small differences in base results probably increase with applying the post-processing. However, the overall results are not too much different. Sorry for the inconvenience. You can consider the results you get as "normal".

Stay tuned for the new method that's build upon BoostTrack. It is implemented and tested with stable package versions :)