search

Parskatt / RoMa

[CVPR 2024] RoMa: Robust Dense Feature Matching; RoMa is the robust dense feature matcher capable of estimating pixel-dense warps and reliable certainties for almost any image pair.
https://parskatt.github.io/RoMa/
MIT License
556 stars 43 forks source link

Should KDE performs on the two images separately? #34

Closed nnop closed 5 months ago

nnop commented 5 months ago

Thanks for the great method which almost solved the WxBS problem.

I noticed symmetric matching is the default. https://github.com/Parskatt/RoMa/blob/50522299a55efc14ed892caaca9e29a1c8b73e12/roma/models/model_zoo/roma_models.py#L152 and KDE is performed on all sampled matches from (H, 2*W) warp results: https://github.com/Parskatt/RoMa/blob/50522299a55efc14ed892caaca9e29a1c8b73e12/roma/models/matcher.py#L475-L486 The positions and warping of image1 and image2 should be independent. So, should KDE performs on the two images separately?

Another question: Since, the sampling is from symmetric matching, will the results contain many near duplicated matches?

Parskatt commented 5 months ago

Kde is typically run after sparse sampling, and works on 4D coords. Symmetric shouldn't affect it.

Parskatt commented 5 months ago

Regarding duplicates, yes this is likely.

nnop commented 5 months ago

Kde is typically run after sparse sampling, and works on 4D coords. Symmetric shouldn't affect it.

Don't know if I described clear. I mean the 4D coords in the other image shouldn't be taken into account for supporting. It not the problem of symmetric. 4D coords in the first (H, W) and the second (H, W) part are both in range [-1, 1]. Wouldn't directly computing the pairwise distance mistake the samples of the other image into account? https://github.com/Parskatt/RoMa/blob/50522299a55efc14ed892caaca9e29a1c8b73e12/roma/utils/kde.py#L6

Parskatt commented 5 months ago

I'm not sure I understand the question unfortunately. Why would we not want to take the other coordinates into account? We simply view the correspondences as 4D vectors. We have a bunch of those, and we want to reduce sampling in regions where we have high density. It doesn't matter where these vectors come from.

nnop commented 5 months ago

Suppose the red dot and the green dot have the same coordinates in Image 1 and 2. Then the matches represented by the yellow line and the blue line should be equal in the 4D match space. Therefore, does the yellow match act as an inhibition to the blue match?

Parskatt commented 5 months ago

No, because all corresps are from A to B even (we switch the ordering internally)

nnop commented 5 months ago

That clears it up for me, thank you for clarifying my misunderstanding of this representation!