search

mihaidusmanu / d2-net

D2-Net: A Trainable CNN for Joint Description and Detection of Local Features
Other
761 stars 163 forks source link

Some image pairs show a large number of incorrect matches #59

Closed tudipffmgt closed 4 years ago

tudipffmgt commented 4 years ago

Hi,

I am working on a very difficult dataset and for most of the image pairs your method works very well! Thank you. But for some other images I get a large number of matches where the majority is visibly incorrect. Clearly, I would prefer that those image pairs show no matches at all. At the moment I am calculating D2-net keypoints and descriptors for every possible image combination. Would there be are smarter way or some tweaks to eliminate the incorrect matched pairs? Example_1 Example_2

Edit: I am not using the multiscale feature at the moment

mihaidusmanu commented 4 years ago

Hello. Are those matches before or after Geometric Verification? I would expect most of them to get filtered in that step.

Are you using any match filtering techniques? You can try using ratio test (in the paper we suggested 0.95 for multiscale but you might want to tune that) - this is not perfect since it eliminates a large amount of good matches as well but it can help in some cases.

Lastly, you can try to use a retrieval system (NetVLAD, R-MAC, DIR, ...) and only match images with close retrieval descriptors - this should also filter out some of the completely unrelated image pairs.

tudipffmgt commented 4 years ago

Hello. Are those matches before or after Geometric Verification? I would expect most of them to get filtered in that step. Are you using any match filtering techniques? You can try using ratio test (in the paper we suggested 0.95 for multiscale but you might want to tune that) - this is not perfect since it eliminates a large amount of good matches as well but it can help in some cases.

At the moment I am using the standard routine explained in https://github.com/tsattler/visuallocalizationbenchmark/tree/master/local_feature_evaluation (modify_database_with_custom_features_and_matches). So it is already after Geometric Verification and for most images I get reasonable results. Could I add the ratio test somewhere in the modify_database_with_custom_features_and_matches.py there?

Lastly, you can try to use a retrieval system (NetVLAD, R-MAC, DIR, ...) and only match images with close retrieval descriptors - this should also filter out some of the completely unrelated image pairs.

Thanks, I could try that.

mihaidusmanu commented 4 years ago

That implementation only uses a mutual NN matcher (see L124): https://github.com/tsattler/visuallocalizationbenchmark/blob/773c593a33998681934e5228005a0e2c380e74b5/local_feature_evaluation/modify_database_with_custom_features_and_matches.py#L124

You would have to replace it by a ratio test / ratio test + MNN matcher. One implementation is available here: https://github.com/mihaidusmanu/torch-feature-matching/blob/4bb6c4a8172af4b28834ffb115c666ae021226ef/feature_matchers.py#L31

tsattler commented 4 years ago

@mihaidusmanu Do you also want to include the ratio test matcher into the localization benchmark code?

mihaidusmanu commented 4 years ago

@tsattler I will make a note and add it later today / tomorrow morning :)!