search

introlab / rtabmap

RTAB-Map library and standalone application
https://introlab.github.io/rtabmap
Other
2.74k stars 780 forks source link

[Feature-Request] ANMS #1127

Closed borongyuan closed 4 months ago

borongyuan commented 1 year ago

This issue encountered when running SuperPoint on OAK should be related to NMS implementation. When I was learning about Kimera-VIO recently I noticed their use of ANMS. It may help improve post-processing of SuperPoint, and can be used for other types of keypoints as well.

matlabbe commented 1 year ago

Having better distributed keypoints can increase robustness of visual odometry and better accuracy. The implementations seem not using the keypoint feature response to select "best" features (like the current NMS implementation for superpoint), they seem only using 2d position of each feature and remove them uniformly. These algorithms could be useful for other detectors, and they could be used by Feature2D::limitKeypoints(), instead of just using keypoint's response value, a ANMS approach could be used instead to get a better distribution of features in the image.

In complementary, it would also be nice to integrate Kimera-VIO (BSD license) as built-in VIO approach in rtabmap standalone, or having a ros example how to integrate their ros package with rtabmap_ros.

borongyuan commented 1 year ago

Keypoint response is used. The algorithm will sort keypoints by strength at the beginning. But it's certainly not the only factor being considered. To obtain a more even distribution, keypoints with higher response may be filtered out. But SuperPoint itself also tends to generate evenly distributed keypoints. Therefore, I think it is possible to make them work together through appropriate parameter configuration or algorithm improvements.

Yep, it should work perfectly for Feature2D::limitKeypoints().