Cannot have multiple realsense pipelines in Jetson Xavier

[smatsui18]

Required Info
Camera Model	D415
Firmware Version	5.12.08.200
Operating System & Version	Ubuntu 18.04.5 LTS
Kernel Version (Linux Only)
Platform	NVIDIA Jetson Xavier
SDK Version	2.31.0
Language	C++

Issue Description

I want to connect to multiple D415s and have multiple pipelines. In my environment, it freezes when accessing the second device.

I am attaching the shortest code that reproduces the problem. This code is based on the following page. https://github.com/IntelRealSense/librealsense/issues/2219

#include <librealsense2/rs.hpp>
#include <iostream>
#include <map>

int main() 
{
    rs2::context ctx;
    std::map<std::string, rs2::pipeline> pipes;

    for (auto&& dev : ctx.query_devices()) {
        std::cout << "loop start." << std::endl;

        rs2::pipeline pipe;
        auto serial = dev.get_info( rs2_camera_info::RS2_CAMERA_INFO_SERIAL_NUMBER );
        std::cout << serial << " ";
        rs2::config cfg;
        cfg.enable_device(serial);
        std::cout << "pipeline ";
        pipe.start(cfg);
        std::cout << "start." << std::endl;
        pipes[serial] = pipe;

        std::cout << "loop end." << std::endl;
    }

    return 0;
}

The console output is as follows.

loop start.
928222060510 pipeline start.
loop end.
loop start.

[MartyG-RealSense]

Hi @smatsui18 Your serial number instruction is not the common way that a multicam serial number retrieval instruction is structured:

auto serial = dev.get_info( rs2_camera_info::RS2_CAMERA_INFO_SERIAL_NUMBER );

It does not include the rs2_camera_info:: part and instead expresses the instruction as something like this:

auto serial = dev.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER);

This format is used by Dorodnic's C++ example for accessing cameras using multiple pipelines, similar to rs-multicam:

https://github.com/IntelRealSense/librealsense/issues/2219#issuecomment-412350887

[smatsui18]

Thank you @MartyG-RealSense for the replying. I tried removing "rs2_camera_info::", but the result seems to be the same.

[MartyG-RealSense]

In the past (though not for most of the past year from the time of writing this) there have been cases where Jetson boards had difficulty with detecting more than one camera. SDK 2.32.5 introduced new multicam handling code, though this improved problems that were related to rs2::pipeline rather than ones resulting from other causes such as specific models of USB hub. As your information at the top of this case states that you are using SDK 2.31.0, it may be worth upgrading your project to a more recent SDK version if that is possible for you.

Prior to the introduction of the multicam improvements in SDK 2.35.2, some RealSense users found that their camera detection success improved if the cameras were inserted in a particular order (e.g Cam 1 and Cam 2, or Cam 2 and then Cam 1). The link below has an example of such a case.

https://github.com/IntelRealSense/librealsense/issues/5828#issuecomment-625671835

I also note that you are using firmware version 5.12.8.200 with SDK 2.3.1.0. Firmware 5.12.8.200 introduced changes in the firmware that meant that it is recommended for use with SDK 2.39.0 or newer.

Downgrading to firmware 5.12.7.100 (the firmware before these changes were introduced in 5.12.8.200) may be more suitable for 2.31.0.

https://dev.intelrealsense.com/docs/firmware-releases#section-d400-series-firmware-downloads

[smatsui18]

Thank you @MartyG-RealSense for the replying.

I have used SDK 2.35.2 on Jetson Xavier before, but there was a problem that the 2D picture and 3D picture were misaligned. Since then, I have been using SDK 2.31.0 with reference to the following page. https://github.com/jetsonhacks/installRealSenseSDK

In RealSense Viewer v2.31.0, I selected Install Recommended Firmware and downgraded the firmware. However, the result did not change.

Has anyone else had many solutions after upgrading the SDK?

[MartyG-RealSense]

There is an official Intel installation guide for Jetson at the link below that has been updated more recently than the JetsonHacks guide.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md

Alternatively, a guide for building librealsense from source code that has worked consistently can be found here:

https://github.com/IntelRealSense/librealsense/issues/6964#issuecomment-707501049

[MartyG-RealSense]

Hi @smatsui18 Do you require further assistance with this case, please? Thanks!

[smatsui18]

Sorry for the late reply. I am still in the process of trying it out and will let you know when I tested.

[MartyG-RealSense]

Thanks very much @smatsui18 for the update. This case will be kept open for a further time period. Good luck!

[smatsui18]

Sorry for the delay in contacting you.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md Updating the SDK and firmware with reference to the above solved the problem of not being able to access multiple devices. However, now another problem occurs.

I am using RealSense as a one-shot device, and I am experiencing the problem of discrepancy between 3D data and RGB data. It's difficult to explain, but for example, if you take three one-shot RGB and depth shots, the depth data is from the third shot, and the RGB data is from the second shot.

This problem has been seen before and occurs in Jetson with versions later than SDK 2.31.0. Has anyone else ever reported this happening before?

[MartyG-RealSense]

It may be related to RGB-depth sync, which is discussed in the link below.

https://github.com/IntelRealSense/librealsense/issues/8643

In one of the links provided in that discussion, a RealSense team member comments on a phenomenon that sounds like what you are experiencing.

https://github.com/IntelRealSense/librealsense/issues/1548#issuecomment-389070550

[MartyG-RealSense]

Hi @smatsui18 Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.