search

YueLiao / CDN

Code for "Mining the Benefits of Two-stage and One-stage HOI Detection"
Apache License 2.0
89 stars 15 forks source link

Evaluation of pretrained model on HICO-Det is lower than reported. #10

Closed boringwar closed 2 years ago

boringwar commented 2 years ago

Hi, I downloaded your CDN-S/B/L pretrained model, and evaluate them locally with your script. The results are all lower than the reported results. Specifically, Model S mAP: 0.3150153102455715 mAP rare: 0.27220402283111156 mAP non-rare: 0.32780309739534524 mean max recall: 0.6390865334740454 Model B mAP: 0.31671305044814135 mAP rare: 0.27124143008976015 mAP non-rare: 0.3302954825032422 mean max recall: 0.6436087461863998 Model L mAP: 0.3196075598370539 mAP rare: 0.2724063291553318 mAP non-rare: 0.3337066287419839 mean max recall: 0.6492344626931886

I'm wondering what caused the inconsistency. what's the version of opencv-python in your environment? I'm using opencv 4.5.1.

YueLiao commented 2 years ago

The reported results are evaluated with batch size 16 (2x8 or 4x4) by the official Matlab script. The inconsistency may be caused by such two aspects.

boringwar commented 2 years ago

OK, I'll check it.