Experimental results on V-COCO

[winnerwu6]

Hello, when will your training and evaluation code on the V-COCO data set be updated? My recent work requires your code very much. I will be very grateful if you can update it! !

[fredzzhang]

Hi @winnerwu6,

Apologies for the delay. I've been occupied by quite a few things in the past few months. I will try to spend some more time to fix it.

Fred.

[winnerwu6]

Hello, we are unable to reproduce the V-COCO experimental values in your paper. When can you update the V-COCO code? We urgently need the correct code and correct experimental values.

[fredzzhang]

Hi @winnerwu6,

I'm still trying to work out the problem. In the meantime, if you need to cite the paper, you can cite the values that are currently reproducible and add a footnote that the codebase is the being cleaned up after being restructured.

Note that the average precision on V-COCO needs to be adjusted as explained here. So for the results in #34, you'd need to adjust the AP for scenario 1 as $57.36 \times 25 / 24 = 59.75$.

Fred.

[hutuo1213]

Hi, we have the same problem. The accuracy of our model on vcoco is 58.58*25/24=61.00 (scenario 1). The accuracy on hico-det is 37.45mAP. Is there another solution to get the correct accuracy? By the way, we have already done a work based on UPT, is there the same problem with vcoco for UPT?

[fredzzhang]

Hi @yaoyaosanqi,

The UPT code shouldn't have any issues.

The VCOCO results in this paper used an older version the codebase, which was extremely messy. After the submission, the code was restructured. I most likely broke something in that process.

[hutuo1213]

Hi,

Our PViC-based work is nearing completion. Is there another way to achieve correct V-COCO accuracy?

Embarrassingly our method is below the 62.8mAP reported by PViC on V-COCO, but higher than PViC on HICO-DET.

[fredzzhang]

Hi @hutuo1213,

Sorry I probably don't have time to fix this issue anymore. You could reference the performance that can be reproduced and compare against that.

Fred.