search

QVPR / Patch-NetVLAD

Code for the CVPR2021 paper "Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition"
MIT License
517 stars 72 forks source link

Detect number of clusters and PCA dimensions from model #10

Closed rajkataria closed 3 years ago

rajkataria commented 3 years ago

Hi,

I'm trying to replicate the NetVLAD scores on the Pitts30k and Tokyo247 datasets but am getting a significantly lower scores than reported. I've modified the code slightly by commenting out the Patch-NetVLAD portions to improve speeds, i.e. I'm not extracting any patches in feature extraction nor am I re-ranking the results.

For example, I'm getting: Recall NetVLAD@1: 0.7031 for Pitts30k Recall NetVLAD@1: 0.3778 for Tokyo247

As reference, you've reported: Recall NetVLAD@1: 0.835 for Pitts30k Recall NetVLAD@1: 0.648 for Tokyo247

I'm using performance.ini which uses the mapillary pre-trained model. I have tried using the pittsburgh pre-trained model, but I get a size mismatch error (copied and pasted below). Any help would be appreciated. Thanks!

Raj

=> loading checkpoint 'C:\Patch-NetVLAD\patchnetvlad./pretrained_models/pittsburgh_WPCA4096.pth.tar' Traceback (most recent call last): File ".\feature_extract.py", line 179, in main() File ".\feature_extract.py", line 165, in main model.load_state_dict(checkpoint['state_dict']) File "C:\Anaconda3\envs\patchnetvlad\lib\site-packages\torch\nn\modules\module.py", line 1051, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Module: size mismatch for pool.centroids: copying a param with shape torch.Size([64, 512]) from checkpoint, the shape in current model is torch.Size([16, 512]). size mismatch for pool.conv.weight: copying a param with shape torch.Size([64, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([16, 512, 1, 1]). size mismatch for WPCA.0.weight: copying a param with shape torch.Size([4096, 32768, 1, 1]) from checkpoint, the shape in current model is torch.Size([4096, 8192, 1, 1]).

Tobias-Fischer commented 3 years ago

Hi @rajkataria, many thanks for your report. As stated in the paper, for the Pitts and Tokyo results we used the model trained on Pitts. The Pitts model uses 64 clusters, whereas the Mapillary model uses 16. In order to run the Pitts model, you'll need to change https://github.com/QVPR/Patch-NetVLAD/blob/3dd248613d159ee2ad9817ac9419f98137abf9f0/patchnetvlad/configs/performance.ini#L19

to 64 and the size error should go away.

Let us know if that solved your issue.

Tobias-Fischer commented 3 years ago

@stephenhausler - let's modify the code to catch that error and print a message saying that probably the num_clusters need to be adapted. Or, even better (not sure if easily feasible) let's try and read the num_clusters, num_PCA etc. from the model so we don't need to specify it.

rajkataria commented 3 years ago

Hi @Tobias-Fischer, thanks for the quick response. I missed the detail regarding the number of clusters. Thanks for pointing it out. It seems to be working now.