Different results in the evaluation

[VanesaLomas]

Hi!

I have been testing your code, and when I run the evaluation script to evaluate the HABBOF dataset I obtain a slightly different result than yours. I have obtained an AP of 95.9 (AP0.5) in contrast with the 98.1 that you obtained. I have run the code over the whole dataset(5837 images) with your pretrained weights(the ones to evaluate HABBOF for a resolution of 1024). Do you have any idea of what can cause this difference in the results?

Thanks in advance! :)

[duanzhiihao]

Hi, I can think of several possible reasons:

• Did you evaluate it at 1024x1024 resolution?
• There are 4 videos in HABBOF. We compute AP for HABBOF by first computing the AP for each video (let's say, we have AP1, AP2, AP3, AP4), and we then average them to get the final AP for HABBOF (ie, AP = 0.25 * (AP1+AP2+AP3+AP4)). Did you compute AP the same way, or did you directly compute AP for all 5837 images in a single step?
• What is your confidence threshold for evaluation? (see below for example) https://github.com/duanzhiihao/RAPiD/blob/bc0f2f049ac376536d8de8bf11af12973ccd18e4/evaluate.py#L45

There may be other factors that I missed. If you can provide your evaluation script I'm happy to help find where causes the difference.

[VanesaLomas]

I found the error: I was computing AP for all the images in a single step, not for each video separately. Computing AP for each video and then computing the average I obtain the same result as you.

Thank you so much for your help!!