search

magicleap / SuperGluePretrainedNetwork

SuperGlue: Learning Feature Matching with Graph Neural Networks (CVPR 2020, Oral)
Other
3.31k stars 672 forks source link

Bad homography transformation obtained from superglue matchpoints. Help! #126

Open SmileyScientist opened 1 year ago

SmileyScientist commented 1 year ago

Hi,

I am using SuperPoint and SuperGlue to extract features and match them to compute homography between two images but the matchpoints seem to be performing worse than SIFT-FAST.

The left image is 1080x1088 and right is 1920x1440. They are captured using two different sensors. frame_1

The matchpoints look good but if I use them to compute homography using cv2.findhomography(), this is what I get frame_1_warp_homography

I have tried modifying a lot of parameters but the homography I get using the matchpoints is no up to the mark. I am using "outdoor" superglue to accommodate for multiple sensors whose intrinsic matrices are unknown.

Here is the wrap using SIFT and FAST which looks waaaaay better frame_2_warp_homography

I want to get a decent homography transformation using SuperPoint and SuperGlue. Any idea on what the issue is and how I can fix it? Note: the goal is NOT to retrain either of the networks.

All help is much appreciated. Thanks!

sarlinpe commented 1 year ago

The correspondences look fine. The issue is likely due to an incorrect use of cv2.findhomography - I've used it in the past without any problem.

foxkw commented 1 year ago

The correspondences look fine. The issue is likely due to an incorrect use of cv2.findhomography - I've used it in the past without any problem.

Why are there so few points that can be matched

SmileyScientist commented 1 year ago

@Skydes , have you used them when the image's keypoints in the world space are not planar? I recently re-learnt that homography will only work when the points are on a planar surface and in my case, I have points being picked from two different planar surfaces.

@foxkw , the threshold for keypoint match is high. I am after high quality matches. Even though I am using a match threshold of 0.95, I still get some false positives. Do you have any experience with this?

foxkw commented 1 year ago

NO. but I found that the number of points you detected is also very small, and I also encountered this situation; I don't know whether this is the cause; I found that the number of matches of the superpoint model I trained with superglue after homography transformation is far less than the official result. I wonder if you have encountered and solved this phenomenon

129