search

LongLong-Jing / Cross-Modal-Center-Loss

Cross-Modal Center Loss for 3D Cross-Modal Retrieval (CVPR2021)
33 stars 9 forks source link

Issue in mAP calculation #11

Open ktertikas opened 2 years ago

ktertikas commented 2 years ago

Hello, thank you for your code!

I am having a look at the way that the mAP is being calculated, in the following function: https://github.com/LongLong-Jing/Cross-Modal-Center-Loss/blob/5025837561a5e059940462edde99aafe80680871/evaluate_retrieval.py#L94

It seems to me that the calculation has a small issue, when you do the calculation for Image2Image, Mesh2Mesh, and Point2Point. In particular, I believe that you should remove the query item from the distance calculations, because otherwise you would always have the diagonal distance matrix value being 0 (it will be the same item in both query and database), so this would produce a slightly wrong mAP value.

Thanks, Konstantinos

piseabhijeet commented 2 years ago

@ktertikas

You are right and i had noticed that too. The code should be changed for the same modalities calculation.