Implement SuperPoint / SuperGlue

[sbucaille]

Model description

The SuperGlue network is a Graph Neural Network combined with an Optimal Matching layer that is trained to perform matching on two sets of sparse image features. SuperGlue is built on top of SuperPoint model which consists of detecting the most interesting keypoints in an image. With the keypoints of two different images, SuperGlue proceeds to the matching.

I noticed there was no image matching models implemented in transformers library so I propose this first one. I extensively used it in other activities and am new to transformers git contributions, so I am willing to implement it myself as a first contribution.

Open source status

• [X] The model implementation is available
• [X] The model weights are available

Provide useful links for the implementation

SuperPoint and SuperGlue code and weights are available at https://github.com/magicleap/SuperGluePretrainedNetwork The original paper of SuperPoint : https://arxiv.org/abs/1712.07629 The original paper of SuperGlue : https://arxiv.org/abs/1911.11763

[arkouda]

I could help here, I have worked with both of these models as well as LoFTR which is the current SOTA in image matching. I am new to Transformers contributions as well, but would love to contribute if any of the repo maintainers agree to the inclusion of these models

[sbucaille]

Hi @arkouda , I was about to use this model as an opportunity to contribute to the project, so I'll be working on the SuperGlue model implementation myself but feel free to take care of the LoFTR :smile: