No depth stream from D405 when D435 is also connected

[badinkajink]

---

Required Info
Camera Model	D405 + D435
Firmware Version	5.15.1
Operating System & Version	Ubuntu 22.04
Kernel Version (Linux Only)	6.8.0-40-generic
Platform	PC
SDK Version	2.54.2
Language	python 3.10.12
Segment	robot

Issue Description

When I connect to a D435 camera and D405 camera at the same time using the pyrealsense2 library, the depth image from the D405 is zeroed out. The D405 color stream is still intact, and the D435 color/depth streams are also unaffected.

However, on the realsense-viewer app, I can see that the color/depth streams for both cameras still work.

Both cameras are on the same firmware version and are plugged into USB 3.2. Disabling the depth stream on the D435 does not affect the D405. I also have tried a hardware reset on the D405.

If I unplug the D435, connect and stream from the D405, and then plug-in and begin streaming from the D435, the D405 color+depth are not affected.

Each camera is wrapped around pyrealsense2 as a Camera object, which has the pyrealsense2.pipe and pyrealsense2.config class variables. Below, I provide an abbreviated snippet of how I am connecting to the cameras:

import pyrealsense2 as rs

class Camera: 
...
def connect(...):
    self.config.enable_device(device_serial)
    self.config.enable_stream(rs.stream.depth, self.w, self.h, rs.format.z16, self.fps)
    self.config.enable_stream(rs.stream.color, self.w, self.h, rs.format.rgb8, self.fps)
    self.pipe.start(self.config)

I appreciate any help or insight.

[MartyG-RealSense]

Hi @badinkajink The RealSense Viewer tool is designed for handling multiple cameras simultaneously. When writing a program script though, you will need to use multiple camera code - the rs.context instruction - otherwise the program may choose only one camera and ignore the other.

The link below has an example of a RealSense multiple camera display program for pyrealsense2 called multiple_realsense_cameras.py

https://github.com/ivomarvan/samples_and_experiments/tree/master/Multiple_realsense_cameras

https://github.com/ivomarvan/samples_and_experiments/blob/master/Multiple_realsense_cameras/multiple_realsense_cameras.py

[badinkajink]

Hi @MartyG-RealSense, thank you for the response.

Yes, my code is based off of those links you sent. I call rs.context and obtain the names and serial numbers for each device connected, after which I instantiate Camera objects with their corresponding serial numbers, as described in the initial post. To clarify, I can stream color from both cameras programmatically without issue, but the depth image is zeroed out for the D405, and only ever the D405.

I plan to debug further today.

[MartyG-RealSense]

Thanks very much for the clarification about your use of the rs.context method.

Does the D405 depth still not appear if you test the pyrealsense2 multicam script at https://github.com/IntelRealSense/librealsense/issues/8388#issuecomment-782395443

I look forward to your next update after debugging. Good luck!

[badinkajink]

Thank you for that link! I suspected that my problem was not related to the cameras, and that script helped me debug the issue (though perhaps I should've been more careful to begin with).

The depth image was always intact, and it was only after passing it to Open3D's RGBD image constructor that the Open3D depth image was zeroed out. This is because I was passing in the D435's depth scale, (0.001), rather than the D405's depth scale, (0.0001).

This occurred because I was calling get_depth_scale on the pipe profile prior to setting the config with a device serial number. By default, it grabbed the depth scale of the first device, which was the D435.

[MartyG-RealSense]

You are very welcome. I'm pleased to hear that you had a successful outcome. Thanks very much for the update!