search

yxgeee / OpenIBL

[ECCV-2020 (spotlight)] Self-supervising Fine-grained Region Similarities for Large-scale Image Localization. 🌏 PyTorch open-source toolbox for image-based localization (place recognition).
https://yxgeee.github.io/projects/sfrs
MIT License
271 stars 41 forks source link

Performances on Oxford and Paris #21

Open amaralibey opened 3 years ago

amaralibey commented 3 years ago

Hello, I tried your pretrained model on cnnimageretrieval-pytorch test script, and I got mAP : 67.90 for oxford (73.9 on your paper) mAP : 76.64 for paris (82.5 on your paper)

Am i missing something ? (both with and without PCA gave similar performances)

yxgeee commented 3 years ago

All the testing configs are following those used in SARE and NetVLAD papers. Please try to find more details in their papers, e.g., the image size. Also note that the normalization parameters (mean & std) used in this repo for training the model may not the same as those used in cnnimageretrieval-pytorch. Please double-check.

Guan2014 commented 3 years ago

@aalibey have you solved the problem? When I set the size as (640, 480), I got the same results as yours. When I keep the original size of datasets, the mAP is 71.49 for Oxford and 79.40 for paris, still lower than the value 73.9 and 82.5 in the paper.

yxgeee commented 3 years ago

@Guan2014 Please follow the setup in https://github.com/Relja/netvlad/blob/master/demoRetrieval.m. For example, use PIL.Image.BICUBIC for resizing.

Guan2014 commented 3 years ago

@yxgeee Thank you! I'll have a try.

Guan2014 commented 3 years ago

@yxgeee I didn't cropped the images with the given bounding box, but used the full image. According to https://github.com/Relja/netvlad/blob/master/demoRetrieval.m, NetVLAD doesn't resize the Oxford/Paris images if using the full image. Could you please tell me the required image size you used for oxford dataset?
In addition, the above results are obtained with feature dimension 4096, and I find a worse mAP with feature dimension 256. There are two mAPs for feature dimensions 4096 and 256 in https://github.com/Relja/netvlad/blob/master/demoRetrieval.m. Could you tell me which dimension you use? Thanks!

xuzhuang1996 commented 2 years ago

Have you get the 82.5% mPA in Paris6K?