Closed ABaldrati closed 3 years ago
Hi,
Thank you for your interest in our work.
To answer your question: Yes, the query (reference) image is removed when computing metrics - this applies to both the Recall and the Recall_Subset metrics.
The evaluation procedures can be found here starting from line 274 to line 299. In particular, the removal of the query image is on line 296.
Please note that the above links point to a different repository that hosts our model (hence, also the evaluation code).
The behavior is consistent with previous work (TIRG).
Great! Thanks @Cuberick-Orion for the prompt and detailed reply!
Hello
I would like to have some clarifications on Table3: “Retrieval performance on CIRR”: In particular I would like to ask whether the query image (a.k.a the reference image) is removed from the retrieved results when calculating metrics.
From the results, particularly the Random (theoretical) row, we can infer this. In fact if the query image was not removed the _Recallsubset@1 should be 1/6 (16.6%) instead of 1/5 (20%).
Before proceeding with my experiments I would like to have a confirmation directly from you.
Thanks for the amazing work