Ubuntu Virtualbox VM - ROS Noetic - D415 RealSense not recognized

[TDVL]

Hello, I have an Ubuntu Virtualbox VM with installed ROS Noetic which works great. I would like to use the D415 RealSense depth camera for RTABMap mapping procedure, however the D415 is not recognized. I have installed the USB 3 extension pack, everything works well, only the D415 is not recognized. What is the problem? Thank you in advance.

[MartyG-RealSense]

Hi @TDVL The RealSense ROS1 wrapper has a launch file that supports RTABMAP.

roslaunch realsense2_camera opensource_tracking.launch

https://github.com/IntelRealSense/realsense-ros/blob/ros1-legacy/realsense2_camera/launch/opensource_tracking.launch#L17

---

An alternative approach is to launch rs_camera.launch first and then the RTABMAP rtabmap.launch launch file afterwards.

roslaunch realsense2_camera rs_camera.launch

roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="-delete_db_on_start" depth_topic:=/camera/depth/image_rect_raw rgb_topic:=/camera/color/image_raw camera_info_topic:=/camera/color/camera_info approx_sync:=false frame_id:=camera_link

[TDVL]

Hello Marty, The RTABMap is OK. The first thing is to launch the D415 in ROS Noetic on Virtualbox Ubuntu VM. In VM the camera is not recognized. Hence, how can I launch the D415 in VM Ubuntu in ROS? Thank you in advance.

[MartyG-RealSense]

My knowledge of getting RealSense cameras working with VMs is limited, unfortunately.

https://github.com/IntelRealSense/realsense-ros/issues/3128#issuecomment-2179959280 has some references about using RealSense with ROS and Virtualbox. One user who could not access their camera found that updating the camera's firmware driver fixed the issue.

The firmware driver version that should be used with SDK 2.50.0 and wrapper 2.3.2 is 5.13.0.50.

[TDVL]

I have installed the specified fimrware. The USB 3 support is turned on, since the USB 3 extension is provided and installed. The ROS works great, and the Ubuntu too. The Virtualbox sees the D415 in its USB 3 filter. I don't know what is the problem.

[MartyG-RealSense]

Is the camera detected by rs_camera.launch if you cancel out of the launch with Ctrl-C after No RealSense devices were found, unplug and re-insert the camera and then repeat the roslaunch instruction?

If it is not detected after an unplug and replug, is it detected if you add initial_reset:=true to your roslaunch instruction to reset the camera at launch?

[TDVL]

Hello, I have unplugged and plugged the camera, also I have reset it. Still, the camera is not detected in VM in ROS

[MartyG-RealSense]

Is the camera still undetected if you perform an RGBD launch with the RealSense ROS wrapper instead of using rs_camera.launch.

1. Install support for an RGBD launch on Noetic with the command below.

sudo apt-get install ros-noetic-rgbd-launch

2. Launch the rs_rgbd.launch launch file.

roslaunch realsense2_camera rs_rgbd.launch

[TDVL]

Hello, Yes, the camera is still not recognized after I have used the roslaunch realsense2_camera rs_rgbd.launch

[MartyG-RealSense]

What method did you use to install the RealSense ROS1 wrapper 2.3.2 please?

[TDVL]

Hello, I have used the sudo apt-get install ros-noetic-realsense2-camera method, as you suggested me previously.

[MartyG-RealSense]

Thanks very much for the confirmation of the command that you used, which installs librealsense 2.50.0 and ROS1 wrapper 2.3.2 at the same time.

I wonder whether the camera would not be detectable if you have enabled USB 3 support with the Virtualbox USB 3 extension but the camera is being mis-detected by the roslaunch as being on a USB 2 connection.

Does it make any difference if you unplug the micro-sized end of the USB cable from the side of the camera, turn it around the other way and re-insert it into the camera (USB Type-C cables are two-way insertion at the micro-sized end).

[TDVL]

Hello Marty, I have managed to start the RealSense in VM, however after 30 minutes the camera stream failed and the message was again: the realsense device not detected. Also, when the camera started in ROS, I have tried the RTABMap, you can see the error message on screenshot. I don't know, maybe the RTABMap did something with the D415 and it does not start now?

[MartyG-RealSense]

You could check whether the camera is overheating and causing a detection failure by touching its casing a few minutes after launch. If it is already hot to the touch after 3 minutes then it would indicate a possible problem with the USB port or the USB cable.

RTABMAP would not be able to do anything to damage the camera.

Have you tried the opensource_tracking.launch launch file as suggested at https://github.com/IntelRealSense/realsense-ros/issues/3170#issuecomment-2254379351 so that RTABMAP support can be enabled in the RealSense ROS wrapper launch without having to use rtabmap_launch

[TDVL]

Hello Marty, The D415 starts again in ROS, however the RTABMap dose not starts with the suggested command: "roslaunch realsense2_camera opensource_tracking.launch". This is the error.

[MartyG-RealSense]

It could be that the opensource_tracking launch file is expecting rtabmap_ros to be installed as a prerequisite, like in Intel's D435i SLAM guide that uses the opensource_tracking launch file.

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i

Can you confirm whether you have rtabmap_ros installed, please?

[TDVL]

Hello Marty, The D415 starts again in ROS, however the RTABMap dose not starts with the suggested command: ||roslaunch realsense2_camera opensource_tracking.launchroslaunch realsense2_camera opensource_tracking.launch"

[MartyG-RealSense]

The D435i SLAM guide is very old and its instructions refer to the ROS1 Kinetic version from around 2016. Melodic and Noetic came after that.

Have you tried installing the versions of the components for Noetic, please?

sudo apt-get install ros-noetic-imu-filter-madgwick
rtabmap_ros: sudo apt-get install ros-noetic-rtabmap-ros
robot_localization: sudo apt-get install ros-noetic-robot-localization

[TDVL]

Hello Marty, I have installed the rtabmap, it starts, however it fails too since the rviz starts and stops. There is an error, in both way. I have tried the opensource_tracking.launch launch file as suggested, and I have tried the the other way too (first launch the realsense, later the rtabmap in other terminal). These are the errors.

[MartyG-RealSense]

You may need to enable the IMU's gyro and accelerometer in the ROS launch if you have not done so already. At a later date after the D435i SLAM guide was published, the IMU topics were made disabled by default but the guide was not updated to reflect this.

roslaunch realsense2_camera opensource_tracking.launch enable_gyro:=true enable_accel:=true unite_imu_method:=linear_interpolation

[MartyG-RealSense]

Hi @TDVL Do you require further assistance with this case, please? Thanks!

[TDVL]

Hello Marty, I have tried with roslaunch realsense2_camera opensource_tracking.launch enable_gyro:=true enable_accel:=true unite_imu_method:=linear_interpolation, but the error remains. The rviz starts and fails in a second. Also, a series of warnings and errors are present.

[MartyG-RealSense]

There is very little documentation available on the Failed to meet update rate warning. When ROS does generate this error though, it sometimes provides this advice:

Try decreasing the rate, limiting sensor output frequency, or limiting the number of sensors.

I believe that 'sensor output frequency' is referring to the IMU frequency.

The RealSense ROS launch is probably already using the accel and gyro minimum FPS speeds though.

Does the number of warnings reduce if you set unite_imu_method in the opensource_tracking launch to copy instead of linear_interpolation?

unite_imu_method:=copy

[MartyG-RealSense]

Hi @TDVL Do you require further assistance with this case, please? Thanks!

[TDVL]

Hello Marty, Yes, I need further help since the RTAB-Map does not start with the unite_imu_method:=copy option. The Failed to meet update rate is still on. The camera works well, I have attached the screenshot from rviz.

[MartyG-RealSense]

I located a case at https://github.com/IntelRealSense/realsense-ros/issues/2652 where Virtualbox is being used with ROS1 Noetic and the error Failed to meet update rate! is also experienced. No data was being published on their /imu topic. There was no solution in that case unfortunately

In that discussion I made the following comment:

---

'Failed to meet update rate' is produced by robot_localization and not by librealsense or the RealSense ROS wrapper. I researched the subject extensively in non-RealSense discussions but it seems that nobody in the ROS community has found a solution to resolve the warnings despite many people having experienced them with robot_localization, unfortunately.

---

I researched again today cases involving 'Failed to meet update' rate and people still did not seem to have solved it. A RealSense user at the link below concluded though that it was being caused by their USB port.

https://support.intelrealsense.com/hc/en-us/community/posts/8338902895379-D435I-IMU-ISSUE

[TDVL]

Hello, OK, thank you. Hence, how should I launch now the mapping? Can you provide me the exact settings based on your research? The camera is working well, the USB3 port is recognized and I can see the pointcloud in rviz. Only the mapping doesn't starts.

[MartyG-RealSense]

The Intel RealSense developer who created the RealSense ROS1 wrapper suggests at https://github.com/IntelRealSense/realsense-ros/issues/1080 that if mapping doesn't start then try using the topic /voxel_cloud and the frame_id map

[MartyG-RealSense]

Hi @TDVL Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.