SLAM with D400 series RealSense cameras

[ChrisHughes]

I have a working prototype D400 (D430) series on Ubuntu 16.04 (USB Device ID 8086:0AD4).

When attempting to run RTabMap (built from master) it will not see the camera in Sources under the RealSense source. The ZR300 camera works well under this build.

I have librealsense2 installed on the system and various examples in there work well. I need some sort of SLAM positioning working, so if there's any code branch I can try or other troubleshooting data I can provide to help get this working I'd be more than happy to help.

Device info: 
    Name                          :   Intel RealSense 430
    Serial Number                 :   728612070061
    Firmware Version              :   05.08.06.00
    Physical Port                 :   /sys/devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1:1.0/video4linux/video1
    Debug Op Code                 :   15
    Advanced Mode                 :   YES
    Product Id                    :   0AD4

[matlabbe]

It seems that new realsense sdk works differently from previous versions (r200, zr300...). Unfortunately, I don't own a D400 series to integrate the new sdk in RTAB-Map standalone. Code for old realsense sdk is here.

However, if you use rtabmap_ros, and that realsense provides a ROS package for D400, it could work out-of-the-box like the various examples shown on this page. Maybe just some topic remaps needed, as I can see in this section, rtabmap could be launched like that:

$ roslaunch realsense_ros_camera rs_aligned_depth.launch
$ roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start" depth_topic:=/camera/aligned_depth_to_color/image_raw rgb_topic:=/camera/color/image_raw camera_info_topic:=/camera/color/camera_info

Not sure about the camera_info topic and if the images are synchronized, but if so, add approx_sync:=false on rtabmap line above.

cheers, Mathieu

[eric-schleicher]

I have a production unit on the way. I'll see if i can get it to work and report back.

[ChrisHughes]

Thanks for the tip on using ROS, @matlabbe

I got it working, however given no RGB camera on the D430 the command used was

roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start" depth_topic:=/camera/aligned_depth_to_infra1/image_raw rgb_topic:=/camera/infra1/image_rect_raw camera_info_topic:=/camera/infra1/camera_info

[eric-schleicher]

@ChrisHughes are you saying the that the 430 lack an RGB sensor? or that you're not using it. the 415 and 435 have RGB is that different from your pre-production 400?

[ChrisHughes]

The 430 does not come with an RGB sensor. You get two stereo IR sensors, so the point cloud generated is greyscale. Here is a comparative chart of the different sensors available in the D400 range. I noticed that the 430 is now listed on Mouser so it appears it isn't just a prototype.

[eric-schleicher]

yup. i see the whole line of modules now. good to know. I'm using external camera and wouldn't mind putting a module and d4 board in my own enclosure.

[CoreyCazes]

How would you go about launching rtabmap_ros with two D400 series cameras?

[matlabbe]

See https://answers.ros.org/question/320708/rtabmap-ros-with-two-realsense-d435-cameras/