IMU data hangs on Raspberry Pi 4

[tinarg13]

---

Required Info
Camera Model	D435i
Firmware Version	05.12.07.100
Operating System & Version	Raspberry Pi OS
Kernel Version (Linux Only)	5.4.51-v7l+
Platform	Raspberry Pi
SDK Version	V2.38.1
Language	C++
Segment	others

Issue Description

My application is working fine, but after a few minutes I see that the data from motion module is hanged up. The depth and color streams are still working during that. After reopen my application everything is working good, but after a few minutes this situation happened again. I thought that maybe is some issue with my code, but I have the same situation on realsense-viewer. After a few minutes I am getting error "No frames Received" only for motion module stream:

After pressed the "Motion module on/off" button everything is working again for a short time. I have re-calibrated the IMU sensor according your description, but I have still this issue.

I have tested my application also on the PC (Ubuntu) and I didn't see this issue, only on the Rasberry Pi 4.

Maybe I should change FPS for IMU sensor? I am using this configuration now: cfg.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F, 63); cfg.enable_stream(RS2_STREAM_GYRO, RS2_FORMAT_MOTION_XYZ32F, 200);

Or maybe another idea?

[MartyG-RealSense]

Hi @tinarg13 There are known issues that can sometimes occur with having the depth, color and IMU streams active at the same time. How the scenario typically manifests is that having IMU and depth or IMU and color active works fine, but problems soon appear if all three are active at the same time.

A solution to this problem in Python is to put the IMU on one pipeline and depth / color on another pipeline. The link below has an approach to this in the Python language that was written by a RealSense user with the same IMU-depth-color problem:

https://github.com/IntelRealSense/librealsense/issues/5628#issuecomment-575943238

A C++ user who tried the same approach found in their own particular case that having a single pipeline and using callbacks worked better:

https://github.com/IntelRealSense/librealsense/issues/6370#issuecomment-626340872

[tinarg13]

Hi Marty,

I changed my application and made two separates pipes for the IMU and depth / color. It is working on my PC, but when I run the same application on the Raspberry Pi then I am getting this error: "Could not open device failed to set power state".

I tried also to use this code from your sample for callback solution:

auto callback = [&](const rs2::frame& frame)
{
    std::lock_guard<std::mutex> lock(mutex);
    if (rs2::frameset fs = frame.as<rs2::frameset>())
    {
        // With callbacks, all synchronized stream will arrive in a single frameset
        for (const rs2::frame& f : fs)
            counters[f.get_profile().unique_id()]++;
    }
    else
    {
        // Stream that bypass synchronization (such as IMU) will produce single frames
        counters[frame.get_profile().unique_id()]++;
    }
};

But I can't get frameset data, only separate frame. Is it some issue with synchronization?

[MartyG-RealSense]

Are you able to have all three stream types active in the Viewer for a period longer than a few minutes on Pi if you reduce the FPS and resolution like the on-screen message on your Viewer image suggests, please?

I would also suggest checking the ASIC Temperature under the Controls section of the Stereo Module side-panel options during the first few minutes. If the temperature is greater than 40 then it would be a cause for concern. If it was over 60 by the time that a few minutes have passed then it would suggest a significant problem with overheating.

In regard to synchronization, IMU and depth / color are not in sync by default. The link below discusses the subject of syncing with the IMU.

https://github.com/IntelRealSense/librealsense/issues/3205

[tinarg13]

Marty,

I used the viewer with my normal FPS and resolution settings for testings. This is the result:

• error after 7 minutes - ASIC temperature 43
• error after 13 minutes - ASIC temerature 47
• error after 2 minutes - ASIC temperature 47
• error after 4 minutes - ASIC temperature 49

After that I reduced the FPS and resolution, but I got the same error after 16 minutes.

Do you have another idea?

[MartyG-RealSense]

47 to 49 is excessive even with all 3 streams active. It is not necessarily the source of the problem though. I ran some tests with SDK 2.38.1 with my own Viewer and D435i at an average camera temperature of late 20s. On my own machine, the problem manifests as No frames received occurring on the RGB stream shortly after all three streams have been activated. It occurred no matter which order the streams were activated in. This is an issue that I have handled in past cases.

I ran the same test in rs-multicam as it can be used with just one camera, and on the D435i it outputs panels automatically for the depth, color and IMU streams. The problems experienced in the Viewer did not occur in rs-multicam when streaming all 3 stream types simultaneously. It may therefore be worth basing the C++ version of your program on the rs-multicam code.

https://github.com/IntelRealSense/librealsense/tree/master/examples/multicam

[tinarg13]

Hi Marty,

I ran also the rs-multicam application on my Raspberry Pi and after 8 minutes I saw that Accel and Gyro values were frozen. Strange that the issue is only for the IMU data sensor.

[MartyG-RealSense]

If you leave the camera motionless and just let it stream without moving the camera, does this have any effect on the length of time before a failure of the IMU stream occurs?

[tinarg13]

It is random. I don't see any difference if it stream without moving the camera.

[MartyG-RealSense]

I got much more stable performance in the Viewer with all 3 streams enabled if I used the standalone version of the Viewer instead of the one installed as part of the SDK installation. I note though that the results achieved on PC and those achieved on Pi when performing the same operation do seem to differ.

[MartyG-RealSense]

Hi @tinarg13 Do you still require assistance with this case, please? Thanks!

[tinarg13]

Marty,

I don't want to waste any more time to find a solution around this issue. I am planning to use the Nvidia Jetson instead Raspberry Pi. Please close this case.

[MartyG-RealSense]

Okay, thank you very much for the update. Good luck with the Jetson.