search

cvg / LightGlue

LightGlue: Local Feature Matching at Light Speed (ICCV 2023)
Apache License 2.0
3.19k stars 296 forks source link

Using LoFTR keypoints ? #76

Open tcourat opened 9 months ago

tcourat commented 9 months ago

Would it be possible to use lightglue in order to match keypoints obtained from LoFTR (or other semi-dense/dense matchers, not only sparse) ?

I know they already provided the matches and not only the detection, but there are still wrongly matched points in most cases and maybe LightGlue could filter them.

Phil26AT commented 9 months ago

Hey, we have never tried this but it might be possible. Given the keypoints from both images, You would need to interpolate descriptors from a dense descriptormap (e.g. the one from SuperPoint). Then you can run LightGlue. Please let me know how well this works, I am also curious :)

tcourat commented 8 months ago

I have tried on a pair of images to interpolate the SP descriptors at LoFTR keypoint location. Seems to work decently I adapted the demo notebook.

From SP+Lightglue : sp2 sp1

From LoFTR + Lightglue (interpolation) : loftr2 loftr1

Can be downloaded there : demo_loftr_sp.zip

I guess re-training with LoFTR kpts would lead to even better results, but if we can use lightglue to filter the outliers from SPpoint its already a good thing (I noticed that loftr can often produce some wrong matches).