search

detectRecog / PointTrack

PointTrack (ECCV2020 ORAL): Segment as Points for Efficient Online Multi-Object Tracking and Segmentation
Other
261 stars 47 forks source link

Wrong results #13

Open phj128 opened 3 years ago

phj128 commented 3 years ago

Hi, when I ran your test on validation, I got such results: Evaluate class: Cars sMOTSA MOTSA MOTSP MOTSAL MODSA MODSP Recall Prec F1 FAR MT PT ML TP FP FN IDS Frag GT Obj GT Trk TR Obj TR Trk Ig TR Tck all 83.87 93.31 90.26 95.18 95.20 92.39 96.86 98.32 97.58 4.46 94.70 5.30 0.00 7777 133 252 152 219 8029 151 10097 109 2187 0002 76.16 90.03 85.22 91.56 91.69 87.16 93.91 97.70 95.77 8.58100.00 0.00 0.00 848 20 55 15 30 903 15 1201 11 333 0006 89.56 97.58 91.89 97.77 97.77 92.75 98.88 98.88 98.88 2.22100.00 0.00 0.00 531 6 6 1 3 537 11 756 7 219 0007 87.86 94.51 93.31 97.97 98.05 94.65 99.38 98.68 99.03 3.75 98.11 1.89 0.00 2244 30 14 80 82 2258 53 3159 29 885 0008 88.39 96.74 91.48 97.51 97.60 91.32 97.98 99.61 98.79 1.03 90.48 9.52 0.00 1021 4 21 9 16 1042 21 1216 15 191 0010 88.54 96.35 92.02 97.06 97.18 92.72 97.84 99.33 98.58 1.36 92.31 7.69 0.00 589 4 13 5 8 602 13 659 12 66 0013 -12.12 -5.56 92.84 -2.78 -2.78 99.30 91.67 49.25 64.08 10.00100.00 0.00 0.00 33 34 3 1 1 36 2 124 6 57 0014 76.24 87.36 88.37 94.22 94.55 88.66 95.64 98.87 97.23 4.72 85.71 14.29 0.00 439 5 20 33 36 459 14 552 9 108 0016 70.40 88.97 79.62 89.09 89.09 79.58 91.13 97.81 94.35 8.13100.00 0.00 0.00 760 17 74 1 37 834 4 783 10 6 0018 88.00 95.14 92.61 95.59 95.66 93.44 96.61 99.02 97.80 3.83 88.89 11.11 0.00 1312 13 46 7 6 1358 18 1647 10 322

image

This is not consistent with the claimed one. sMOTSA and MOTSA are similar, but IDS is 152. Is there anything wrong with the test process or the model is not right? Thank you for your awesome work.

anirudh-chakravarthy commented 3 years ago

Hi,

I got the same result using the pre-trained weights. Were you able to solve this?

@detectRecog could you please explain why this performance difference is observed?

phj128 commented 3 years ago

No

EchoAmor commented 3 years ago

@detectRecog I got the same problem ,mine is as follows, why there appeared -12 ?? can u please explain this situation? Screenshot from 2020-11-03 19-48-38

Ilyabasharov commented 3 years ago

@EchoAmor @phj128, Did u carefully examine the 13th track, which consists of much more pedestrians than cars? so, your model often makes mistakes on this track. Look at precision metric, lots of false positives.

EchoAmor commented 3 years ago

@Ilyabasharov oh,thanks a lot! I didn't check the images before

Ilyabasharov commented 3 years ago

@EchoAmor How did you achieve such results ? Here https://github.com/detectRecog/PointTrack/issues/15 you wrote that u have lower results.

EchoAmor commented 3 years ago

@Ilyabasharov this result is run with the author's pretrained weight, and #15 is trained from the initial by myself.

qa276390 commented 3 years ago

Hi @EchoAmor! did you get the result of IDS=21 they report on paper?