Visual code in open3d：view_keypoint.py

[wangzixiang99]

Hello，Why do you need to enter keypoints in the view_keypoint code for visualization? Isn't keypoints automatically obtained? I can't run this view_keypoint code normally

[hanzheteng]

Have you compiled this project successfully?

Basically, view_keypoint.py is just a visualizer from Open3D. It first executes the C++ code for keypoint detection (where the detected point cloud will be saved as a file), and reads the saved pcd files for Open3D visualizer.

Please post any error messages that you may have.

[wangzixiang99]

Have you compiled this project successfully?

Basically, view_keypoint.py is just a visualizer from Open3D. It first executes the C++ code for keypoint detection (where the detected point cloud will be saved as a file), and reads the saved pcd files for Open3D visualizer.

Please post any error messages that you may have.

Thank you very much. This problem has been solved. I use your algorithm to register my dataset, and the result is not very good. My data is ground point cloud data, which has more RGB information and less point cloud information. Is your algorithm mainly used for registration based on the found keypoints? Which registration algorithm do you use? For this form of data, which parameters do you think can be adjusted to better adapt to it?

I'm sorry to have taken up your time. I look forward to your reply.

[hanzheteng]

Registration is one of the potential applications of this keypoint detector, but not the ideal one. Because this method has been optimized for speed (real-time execution). If you do not have concerns of time (which is the case of point cloud registration), then other methods/ways to integrate RGB information may perform better. (See the 6. Future Work section in the README file for more discussions.)

We are using TEASER++ for registration. (RANSAC is not a good idea.) Please see 3.2 Steps in the installation section for more information.

For registration tasks, you may set a smaller threshold for centroid_rgb, and maybe geometric centroid distance as well if needed. In this way, you can obtain more keypoints for downstream applications.

[wangzixiang99]

Registration is one of the potential applications of this keypoint detector, but not the ideal one. Because this method has been optimized for speed (real-time execution). If you do not have concerns of time (which is the case of point cloud registration), then other methods/ways to integrate RGB information may perform better. (See the 6. Future Work section in the README file for more discussions.)

We are using TEASER++ for registration. (RANSAC is not a good idea.) Please see 3.2 Steps in the installation section for more information.

For registration tasks, you may set a smaller threshold for centroid_rgb, and maybe geometric centroid distance as well if needed. In this way, you can obtain more keypoints for downstream applications.

You have also studied the color_icp registration algorithm in open3d before . Why do you use TEASER++ for registration instead of the color_icp method. Isn't it possible to make better use of geometric and rgb information?

(https://github.com/hanzheteng/color_icp--------Colored Point Cloud Registration Revisited, Jaesik Park, Qian-Yi Zhou and Vladlen Koltun, ICCV 2017)

I also used the color_icp algorithm of your recurrence, but the effect of using your algorithm on my dataset is worse than that in official open3d. I don't know why this problem occurs.

[hanzheteng]

Because they are two different approaches for point cloud registration.

• For ICP-based algorithm, this is a family of iterative methods. Correspondences are established by searching for the closest points in the other point cloud (this also requires initial guess of the relative pose). They do not take any keypoints and use all of the points in the point cloud.
• For TEASER++, it assumes that correspondences are established already (and these correspondences are usually sparse; TEASER cannot handle dense cloud due to intractable time complexity). This falls into a typical 4-step registration pipeline: extract keypoints (obtain a sparse cloud), apply feature descriptor, match correspondences from described descriptors, and estimate transformation from correspondences. TEASER++ is the best algorithm so far for the last/4th step, when correspondences are known.

As for parameter tuning for color_icp, please open an issue in that repository and I would be happy to help you there.