Questions about the matchability loss

[Hui-design]

Hi. Thank you for your amazing work! Your design for overlap scores is wonderful. But, I would like to discuss with you the design of the matchability score. You use the virtual labels generated by the feature space to supervise the 33rd or 34th dimension of the output, which is a very amazing idea. However, in the process of reproducing your paper, I found that the saliency_precision (matchability_precision) on the validation set is only 0.4, which means that the network fails to learn the matchability score. I guess there are two possibilities: one is that I reproduce the error, if so, can you provide the log file? The other is to say that this tightly coupled way of descriptors and detectors is not easy to learn?

[ShengyuH]

hi,

Thanks for your interest in our work and good question.

• " I found that the saliency_precision (matchability_precision) on the validation set is only 0.4," Yes this could be the case.
• If you check D3Feat/D2Net, they manage to learn a meaningful detector by explicitly enforcing the hand-crafted patterns in descriptors, so I think "tightly coupled way of descriptors and detectors" is still achievable.
• In PREDATOR, we imagine that there are some patterns(sharp/smooth feature) in descriptors that will naturally emerge and tell us something about matchability(not saliency because it's hard to say what patterns are good for matching, being salient might be the case but it might not be the only answer). In my opinion, compared to their detector loss, our designed matchability loss is more flexible but provides weaker supervisions. I'd like to see any other ideas in the direction that can work better.

@zgojcic Maybe you have something to add?

Best, Shengyu

[Hui-design]

Thank you very much!

[zgojcic]

I agree with the intuition that Shengyu provided above. Matchability is actually a difficult concept for the network to learn, as it probably also relies on the global context of a specific point cloud pair. In Predator, this was not our main target of interest and hence we went with a solution that delivered reasonable results, while being motivated on our intuition of matchability. I am positive that there are better ways to learn matchability and that it is an important concept in point cloud registration. It would be great to see some more principled ideas emerge in the near future.

Best, Zan

[ShengyuH]

Close due to inactivity.

[Hui-design]

Hello, I am a novice in scientific research, and predator has been my study object for nearly half a year. I have ideas and experimental results about matchability, and I would like to consult you for some advice. Can you send me your personal email if you are interested? You can send it to my email: 2225705604@qq.com