search

XuyangBai / D3Feat

[TensorFlow] Official implementation of CVPR'20 oral paper - D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features https://arxiv.org/abs/2003.03164
MIT License
261 stars 38 forks source link

Matching point clouds with different scales #39

Closed xin-peng closed 2 years ago

xin-peng commented 3 years ago

Hello, Thanks for your group's excellent work! I want to use D3Feat to match 2 point clouds which are generated by 3D reconstruction, which means their scales will be different. I am wondering if D3Feat works for matching point clouds with different scales? Thanks!

XuyangBai commented 3 years ago

Hi Thanks for your interest in our work. D3Feat is designed for rigid transformation and I think the performance on point clouds with different scales might not be good because

  1. For both source and target point cloud, the descriptors are extracted with the same receptive field (or the neighborhood of the same size in another world). But for point clouds with different scales, the receptive field should be different according to the scale.
  2. During training, we did not feed the network with point pairs with different scales, the network is asked to predict similar descriptors only for point pairs with exactly the same scale and same local geometry.

So in general I think D3feat may have some generalization ability to small scale changes but may not work for large scale variance.

Best, Xuyang.

xin-peng commented 3 years ago

Hi Xuyang, thanks for your reply! That's fine. I am considering to manually adjust the point clouds' scales to reduce the difference.