Open YvonneZhao0305 opened 2 years ago
You can send the image with semantic color instead of the real RGB image, however, visual loop closure detection won't work. If you have already point cloud with semantic information, a hack would be to feed the point cloud (subscribe_scan_cloud=true). That cloud could be XYZRGB format or XYZI format. The semantic may be encoded in the "RGB" or "I" part of the point cloud.
You can send the image with semantic color instead of the real RGB image, however, visual loop closure detection won't work. If you have already point cloud with semantic information, a hack would be to feed the point cloud (subscribe_scan_cloud=true). That cloud could be XYZRGB format or XYZI format. The semantic may be encoded in the "RGB" or "I" part of the point cloud.
Using voxblox++ for reference, I combine semantic segmentation and geometric segmentation. For each frame of RGB and depth image input, the segments (point cloud with label and specific color) of the current frame will be obtained, and the smaller segments will be filtered out.
Since the segmentation module will take a long time, I expect that the running speed of the system will slow down when combined with RTAB-Map, and the input of low frame rate (point cloud or rgbd image) will also affect the localization effect. Is there a better solution?
It depends on how slow the system get. RTAB-Map updates by default at 1 Hz, so if your segmentation can be done under 1 second, there would be not really a slow down for localization.
You can send the image with semantic color instead of the real RGB image, however, visual loop closure detection won't work. If you have already point cloud with semantic information, a hack would be to feed the point cloud (subscribe_scan_cloud=true). That cloud could be XYZRGB format or XYZI format. The semantic may be encoded in the "RGB" or "I" part of the point cloud.
Hello, is there a way to input semantic images and use loop detection? I want to use loop detection to merge multiple sessions of the semantic map. I hope I can get your help. I've been stuck here for a long time. THANKS @matlabbe
Hello, I want to input the results of instance segmentation and geometric segmentation (pointclouds with semantic information) into rtabmap, but I find that the input of rtabmap is RGB image and depth image. If the pointclouds with semantic information is projected onto a two-dimensional plane and then transferred to rtabmap, it may lose semantic information (not tried yet). What is a good way to transfer semantic information to rtabmap? @matlabbe Thanks.