search

hitachi-rd-cv / qpic

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"
Apache License 2.0
131 stars 33 forks source link

For V-COCO result reproduction, which evaluation code should I use? #7

Closed weiyunfei closed 3 years ago

weiyunfei commented 3 years ago

I tried to reproduce your results on V-COCO, and I encountered some problems. I found that the official evaluation code can only achieve 56.5 for Scenario 1 and 58.6 for Scenario 2 with your pre-trained model, while with the vcoco_eval.py in your project which is in PPDM style, the result is 58.35. However, the evaluation process in vcoco_eval.py seems like Scenario 2 in the official evaluation. Which code should I use to reproduce your experiments on V-COCO? Moreover, if I need to use the vcoco_eval.py, what should I do to get the evaluation results for both scenarios?

tamtamz commented 3 years ago

You should conduct the official evaluation. However, the original evaluation code shows mAP including the action "point", which has to be excluded for the evaluation. For this exclusion, you can use the code provided here by the authors of iCAN.

weiyunfei commented 3 years ago

Thanks! I got the correct results :)