yewzijian
commented
2 years ago Hi @qinzheng93, you are correct in the observation. I used a smaller augmentation because the network considers the point positions (through the positional encoding in the attention layers). The network can use this information to infer certain properties which are not possible in local-only descriptors, e.g. a region might be empty because it is occluded by points nearer to the camera.
I recently performed some tests to understand the behavior against larger transformation perturbations of our trained model. On the 3DMatch test set, compared to e.g. predator, our model performs better for pairs with smaller transformation differences, but worse for pairs with large transformation differences. This may be a result of the weaker augmentation, but also may be due to training data bias (there's more training pairs with smaller transformations).
qinzheng93
Thanks for the great work. I notice that RegTR adopts a much weaker augmentation than the commonly used augmentation in [1, 2, 3]. How does this affect the convergence of RegTR? And will the weak augmentation affect the robustness to large transformation perturbation? Thank you.
[1] Bai, X., Luo, Z., Zhou, L., Fu, H., Quan, L., & Tai, C. L. (2020). D3feat: Joint learning of dense detection and description of 3d local features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6359-6367). [2] Huang, S., Gojcic, Z., Usvyatsov, M., Wieser, A., & Schindler, K. (2021). Predator: Registration of 3d point clouds with low overlap. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition (pp. 4267-4276). [3] Yu, H., Li, F., Saleh, M., Busam, B., & Ilic, S. (2021). Cofinet: Reliable coarse-to-fine correspondences for robust pointcloud registration. Advances in Neural Information Processing Systems, 34, 23872-23884.