magicleap / SuperGluePretrainedNetwork

SuperGlue: Learning Feature Matching with Graph Neural Networks (CVPR 2020, Oral)
Other
3.24k stars 663 forks source link

question about the score matrix for optimal transportation #127

Open why-scholar opened 1 year ago

why-scholar commented 1 year ago

Equation 7 computes the score matrix through the inner product of two vectors, which may cause the score matrix to contain negative values. (1) One of my questions is whether negative values in score matrix have effect in optimal transportation? (2) The values computed based on inner product mean that large values indicate similar keypoints, while small values indicate dissimilar keypoints. However, the sinkhorn algorithm searches the lowest cost for optimal transportation problem. If large values in score matrix indicate better matching performance, it is contradictory with optimal transportation problem. This confuses me a lot, and its my pleasure if you can explain it.

sarlinpe commented 1 year ago

$\bar{\mathbf{S}}$ (eq 7 & 8) corresponds to the scores of the optimal transport in log space. The scores of the classical problem are $\exp(\bar{\mathbf{S}})$. See the code for reference:

https://github.com/magicleap/SuperGluePretrainedNetwork/blob/ddcf11f42e7e0732a0c4607648f9448ea8d73590/models/superglue.py#L259-L265

why-scholar commented 1 year ago

Thanks for your reply. I am still confused about the two mentioned problems. In my opinion, the computation of optimal transport in the log space is not proposed to deal with negative values in the score matrix. In addition, the question about the inner product also confuses me.