search

NVIDIA-ISAAC-ROS / isaac_ros_visual_slam

Visual SLAM/odometry package based on NVIDIA-accelerated cuVSLAM
https://developer.nvidia.com/isaac-ros-gems
Apache License 2.0
862 stars 139 forks source link

load_map_and_localize not work on non-horizontal surfaces #137

Open tuch77731 opened 9 months ago

tuch77731 commented 9 months ago

camera: realsense d435i isaac_ros_visual_slam version: 2.1 jetson orin nx

I set up the camera and looked at the ceiling base_link to camera_link tf: 0, 0, 0 , 1.57, -0.785, 0 (xyz, rpy)

parameters: force_planar_mode: False base_frame: base_link input_base_frame: base_link

After I run for a while, I saved map. Next, I load_map_and_localize that Input is localize_near_point. I found some issue.

First, the coordinate of localize_near_point is base on camera_link not on base_link. For example, when robot is on (5, 0, 0), I set (5,0,0) is always False , and It's work I must set (3.5, 3.5, 0). It would be more intuitive if base_link could be set.

Second, localize_near_point is not work when camera_link is not horizontal to the initial point. It's work about -2<z<2 and rx,ry<0.1. Other situations are failures. For example, robot was on (0, 5 ,0), so that camera_link was on (0, 0, -5). The localize_near_point was always failure when I set (0, 5, 0) or set (0, 0, -5) or other value. The other example, I localize_near_point success when my robot was on xyz(0, 0, 0) and rpy(0, 0, 0). Next i spin my robot to xyz(0,0,0) and rpy(0, 0, 1.57)(Camera_link will get some values of rx, ry) . The action localize_near_point is always failure when i use localize_near_point (0,0,0). The reason i think is I cannot specify the initial direction when I set localize_near_point.

swapnesh-wani-nvidia commented 9 months ago

The pose for LoadAndLocalize action is aligned with the base_link frame (x - forward, y - left, and z - up). See the tutorial here.

I don't understand how your camera and base_link is attached. If your robot is at 5 m in the Y direction in map frame how can camera be at -5 in Z axis?

For example, robot was on (0, 5 ,0), so that camera_link was on (0, 0, -5).

Also, for the LoadAndLocalize to work, if the current features from the viewing angle don't match the stored landmarks, it will fail.

tuch77731 commented 9 months ago

I show my experiment. Everything is fine when the camera is mounted horizontally However, when installed at an angle, it will be unable to be LoadAndLocalize in certain positions.

486825107143000631

thank you

Seongoo commented 8 months ago

I have the same issue regarding the third experiment to localize my robot using a RealSense camera on a mapped surface parallel to the xy-plane. If I set my robot at the position (0, 0, 0) and try to localize the robot facing towards the starting point, where the robot looked at when I started mapping, localization was done successfully by giving a localize_near_point (0, 0, 0). However, localization always failed by giving the localize_near_point (0, 0, 0) after only rotating my robot over ±90 degree, even though the transition did not change at all. Would adding a rough robot's orientation for localization solve such an issue?