search

QVPR / Patch-NetVLAD

Code for the CVPR2021 paper "Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition"
MIT License
525 stars 74 forks source link

Cross-match between patches #82

Closed euncheolChoi closed 1 year ago

euncheolChoi commented 1 year ago

Hi. If I have different patch feature maps for, say, 2x2, 5x5, and 8x8 sizes, is there a cross-match result between local patch descriptors created from patches of different sizes? I would be grateful if you could share the results of such experiments.

Also, I think I need to modify the def compare_two_spatial() function in patch_matcher.py to implement such cross-mathcing. Could you shed some insight on the implementation of cross-matching?

Thanks.

Tobias-Fischer commented 1 year ago

@StephenHausler could you please comment on this issue?

StephenHausler commented 1 year ago

Hi @euncheolChoi

In our current implementation, no there is no cross-match between different patch sizes. Each patch size is kept separate.

We've never released any results for such an ablation, but we did actually run this experiment (but I don't have the numbers on hand sorry). It resulted in slightly worse recall results typically. We found that cross-matching between different patch sizes tends to increase the localization window of matches, as it encourages local feature matches between different spatial scales across images. For 6-DoF pose estimation, this is ideal, but for purely visual place recognition this tends to result in best matches that are outside the ground truth radius for localization.

Cross matching can be added to the code by stacking all the features and keypoints across the patch sizes, then just running torch_nn using the stacked features. So you would remove the for loop over patch sizes, then just call torch_nn once for the stacked features then run the rest of spatial/ransac as per normal.

Hope this helps. Feel free to reopen or comment if you have further queries.