search

Parskatt / DKM

[CVPR 2023] DKM: Dense Kernelized Feature Matching for Geometry Estimation
https://parskatt.github.io/DKM/
Other
378 stars 28 forks source link

Unexpected Match Point with Transparent Background #63

Closed qidihan closed 2 months ago

qidihan commented 2 months ago

Thank you for your outstanding work!!! I'm still working on my final project, where I'm trying to estimate the rotation angle between two identical objects. Initially, I tried using the SIFT algorithm, but the results were terrible. Therefore, I'm exploring deep learning methods to calculate the angles. However, I'm encountering an issue that I hope you can help me with. I'm experiencing an issue where I get a match point regardless of the background color I set. The original image on the left has no background, but it seems like the algorithm is automatically adding a background. I'd like to know if there's a parameter to avoid this behavior. Steps to Reproduce:

  1. Load the original image with a transparent background, I tried with opencv and PIL.
  2. Set the background color to any value (e.g., white, black, etc.)
  3. Run the algorithm with this error

Thanks Qidi 111

Parskatt commented 2 months ago

Cant say wxactly whats wrong, but unfortunately dkm/roma are not got at big rotations (in your case 90 degrees). That might be the issue.

Parskatt commented 2 months ago

Regarding background matches, it might be due to the balanced sampling. This will try to get as diverse matches as possible, which sometimes leads to these types of issues.

qidihan commented 2 months ago

Thank you for your answers. I will try other methods.

Parskatt commented 2 months ago

@qidihan you may want to try out our recent work https://github.com/georg-bn/rotation-steerers that works much better for large rotations.

qidihan commented 2 months ago

I tried. Yes, steerers got much better result on rotation. I'm trying export the file with onnx and tensorRT right now. Thanks for your suggestions!!!