search

verlab / accelerated_features

Implementation of XFeat (CVPR 2024). Do you need robust and fast local feature extraction? You are in the right place!
https://www.verlab.dcc.ufmg.br/descriptors/xfeat_cvpr24
Apache License 2.0
1.02k stars 113 forks source link

Multi-modality? #1

Closed adamzenith closed 6 months ago

adamzenith commented 7 months ago

Great work on this project, it looks very promising! What are your thought on using this architecture to find keypoints and descriptors across modalities? This paper attempts to match features across rgb and infrared images. Do you see any reasons why this might not work for your architecture?

guipotje commented 7 months ago

Hello @adamzenith, thank you for the interest in our work!

I believe our architecture is fully compatible with their training pipeline. With minor adjustments, it should be possible to train XFeat with different image modalities as described in XoFTR. However, as we distill keypoints from ALIKE, it is recommended to use self-supervised loss for the keypoints (such as ALIKE itself or SuperPoint), or distill from ALIKE in RGB and enforce repeatability on thermal images for example, to get better results in the sparse setting. Stay tuned for the release of our training code!

guipotje commented 6 months ago

Closing the issue. If you need further clarifications, feel free to re-open it!