search

Parskatt / DKM

[CVPR 2023] DKM: Dense Kernelized Feature Matching for Geometry Estimation
https://parskatt.github.io/DKM/
Other
385 stars 28 forks source link

np.random.choice for good_samples #4

Closed ufukefe closed 2 years ago

ufukefe commented 2 years ago

Hi Johan, thanks for your excellent work!

I wonder why np.random.choice is used for selecting good_samples instead of ordering the confidences. Is it for selecting more sparse keypoints?

Parskatt commented 2 years ago

Hi, we found ordering confidences to be problematic as spatially small areas may contain almost all very confident matches. Typically we want to spread out matches to give good estimation results. Although I'm quite sure that it's possible to do better with the sampling than what we did ;)

ufukefe commented 2 years ago

Thanks for the explanation:)

oguz-hanoglu commented 12 months ago

Thanks for the discussion. This sounds related to the motivation behind "minDistance" of OpenCV's goodFeaturesToTrack. https://docs.opencv.org/3.4/dd/d1a/group__imgproc__feature.html#ga1d6bb77486c8f92d79c8793ad995d541

Parskatt commented 12 months ago

I've later learned this is was also discussed in Overlap Predator (pointcloud registration).

I think LoFTr doesnt have this issue since they have few enough corrs and at coarse stage select all over thresh.

oguz-hanoglu commented 12 months ago

I was not expecting such a quick response :) Actually, I was not expecting a response to an old issue. Thanks for the great work.

Using DKM_v1 on thousands of images, I noticed that it(+cv2.findHomography) sometimes generates inappropriate matrices. I suspect this is the case.

May I ask if DKM_v3 is more stable than DKM_v1 in this sense? I am looking for some good reasons to update "old but good" DKM_v1 :)

Parskatt commented 12 months ago

If you are mostly working on planar scenes I think there is not that big of a difference with v3. The main improvements are for non-planar scenes.

oguz-hanoglu commented 12 months ago

Waov, thanks for the inside information. Just to clarify, a scene as in the image below(just consider the road part), even if is not perpendicular to the camera, does it count "planar" in terms of DKM?

image

Parskatt commented 12 months ago

Yes this would work well I think. By planar I mean scenes/two-view which can be well represented by homographies (which happens either when the 3D structure closely resembles a plane, or when the camera motion is only rotational)

From: oguz-hanoglu @.> Sent: Thursday, September 21, 2023 11:52:13 AM To: Parskatt/DKM @.> Cc: Johan Edstedt @.>; Comment @.> Subject: Re: [Parskatt/DKM] np.random.choice for good_samples (Issue #4)

Waov, thanks for the inside information. Just to clarify, a scene as in the image below(just consider the road part), even if is not perpendicular to the camera, does it count "planar" in terms of DKM?

[image]https://user-images.githubusercontent.com/99868596/269569981-9cf7e05c-80fc-4395-a2c9-93cda5d0d326.png

— Reply to this email directly, view it on GitHubhttps://github.com/Parskatt/DKM/issues/4#issuecomment-1729238794, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFIIB7TPLWOYTWRDGTY7OS3X3QE43ANCNFSM54PYWDJQ. You are receiving this because you commented.Message ID: @.***>

oguz-hanoglu commented 12 months ago

Well, my use case is a single camera taking images every 5 minutes or so. I need to register new image to the previous image and the camera is looking to a scene like this.

So I think I should not expect a performance increase in this scenario by moving from DKM_1 to DKM_3. I should better focus on why my DKM_v1 fails(generates too rigorous rotation etc.) and why it does not fail when I run it again (even if I rerun 10K times) on the same images.

Parskatt commented 12 months ago

You may be interested in some recent homography solvers that are better than opencv. For example the pentagon one