search

KunpengLi1994 / VSRN

PyTorch code for ICCV'19 paper "Visual Semantic Reasoning for Image-Text Matching"
288 stars 47 forks source link

Question about the gain in the paper. #6

Closed yang-yang-ai closed 4 years ago

yang-yang-ai commented 4 years ago

In paper, you said you improve 5.8% on caption retrieval and 12.6% on image retrieval. "our VSRN improves 5.8% on caption retrieval (R@1) and 12.6% on image retrieval(R@1) relatively (following the same strategy [23] of averaging predicted similarity scores of two trained models)." But the table 3 the numbers are not consistent with your claim. " SCANECCV018 [23] 67.4 90.3 95.8 48.6 77.7 85.2 VSRN (ours) 71.3 90.6 96.0 54.7 81.8 88.2"

Your results only 3.9% better on caption retrieval R@1 and 6.1% better on image retrieval R@1. Could you tell me how you get the numbers.

Thanks.

KunpengLi1994 commented 4 years ago

Hi,

Please note that we follow SCAN (ECCV'18) to report the relative improvement, which is calculated by (P_our - P_sota)/P_sota. P_our is the performance of the proposed method and P_sota is the state-of-the-art performance (like SCAN). This is the same way how SCAN calculate the relative improvement reported in their abstract.