search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.43k stars 4.8k forks source link

T265 stops sending data #6362

Open dschnabel opened 4 years ago

dschnabel commented 4 years ago
Required Info
Camera Model T265 & D435
Firmware Version T265: 0.2.0.926, D435: 05.12.03.00
Operating System & Version Raspbian 10.3
Kernel Version (Linux Only) 4.19.97-v7l+ armv7l
Platform Raspberry Pi 4
SDK Version 2.33.1
Language C++/ROS
Segment Robot

Issue Description

After upgrading from SDK 2.31.0 to 2.33.1 my T265 stops sending pose data when running for some time (1-5 min). This happens if the T265 is run together with the D435 and also if I run the T265 alone (less frequently).

This wasn't a problem in 2.31.0 but I was experiencing crashes in 2.31.0 so I decided to upgrade to the latest stable version (2.33.1).

In 2.33.1 I see no more crashes but I seem to be experiencing a very similar problem as described in https://github.com/IntelRealSense/librealsense/issues/5509#issuecomment-587422436.

I saw some discussion on missing serial numbers but in my case both cameras seem to get detected properly (see my ROS startup logs for T265 and D435).

When I monitor the odom topic from the T265, after a while I don't get any more pose (odom) data:

$ rostopic hz /odom

average rate: 200.380
    min: 0.000s max: 0.099s std dev: 0.01482s window: 50000
average rate: 200.380
    min: 0.000s max: 0.099s std dev: 0.01482s window: 50000
average rate: 200.380
    min: 0.000s max: 0.099s std dev: 0.01482s window: 50000
average rate: 200.315
    min: 0.000s max: 0.099s std dev: 0.01483s window: 50000
no new messages
no new messages
no new messages
no new messages
no new messages

For completeness here are my two ROS launch files:

Is there any fix for this?

nixinator commented 2 years ago

nice work and power fluctuation debugging, nice. can i ask what power meter you are using in your screenshot, probably a essential item for robot builders.

HanDaSeul commented 2 years ago

nice work and power fluctuation debugging, nice. can i ask what power meter you are using in your screenshot, probably a essential item for robot builders.

@nixinator My bad, it's a photo not a screenshot😂 It's j7-c. When I plug T265 through that usb tester, placing between RPi usb port and T265, t265 is always recognized as being connected to usb 2.0, not 3.0.

csr-kick commented 2 years ago

@HanDaSeul When I did my testing the RPI4 (4GB) was sitting on my desk with just the T265 plugged in. It happens more frequently when you stress the CPU, and if you stress the USB by also taking in the video streams. When I used a powered USB hub with the D453i it also crashed more frequently, but as you pointed out not all usb hubs can provide the right power. I'll do a quick check with a custom USB hub I have somewhere around here that has its own 5v/5A regulator on it. Can you tell me what USB power analyzer you have? Looks pretty nice.

As we talked earlier in this thread, I wrapped the driver in code that tracks the last pose, restarts the T265 pose stream, then applies the old pose as an offset to the now zero'd poses. Some other trickery with matching depths or lidar scans will relocalize any drift pretty well.

HanDaSeul commented 2 years ago

@HanDaSeul When I did my testing the RPI4 (4GB) was sitting on my desk with just the T265 plugged in. It happens more frequently when you stress the CPU, and if you stress the USB by also taking in the video streams. When I used a powered USB hub with the D453i it also crashed more frequently, but as you pointed out not all usb hubs can provide the right power. I'll do a quick check with a custom USB hub I have somewhere around here that has its own 5v/5A regulator on it. Can you tell me what USB power analyzer you have? Looks pretty nice.

As we talked earlier in this thread, I wrapped the driver in code that tracks the last pose, restarts the T265 pose stream, then applies the old pose as an offset to the now zero'd poses. Some other trickery with matching depths or lidar scans will relocalize any drift pretty well.

@csr-kick If i'm right, USB power analyzer = power meter = 'j7-c'.

As you mentioned about stress, I tried overclocking little bit higher and making sure keep cool(didn't exceed 50 celsius degree), but nothing changed. Through the whole test myself, conclusion is 'as long as you are using RPi, don't try streaming in ROS'. But just realsense-viewer, you can try streaming and there's no problem with connection. I still can't understand why. Only I can make it clear is there's difference of using resources like cpu, power whatever. Before I changed the usb hub to 'CR113', i tried other powered usb hub. Some bad powered usb hubs are not stable and have back-power issue.

I don't have D453i, or kinda depth. That's why I'm trying make wheel odometry model for my project.

Construkted-Reality commented 2 years ago

I don't think it's a power delivery issue. At least not from my tests: I'm running only the T265 attached to the Pi 4 (8gb). No other USB devices. I'm running a simple python script that outputs to the screen the T265 pose values to the screen. There is minimal CPU load.

The Pi is sucking back around 0.8-0.9A with the T265 running. The USB power bank I'm using is capable of delivering 2.5A, so there's no bottleneck there. The T265 still stops at random intervals...

image

HanDaSeul commented 2 years ago

@Construkted-Reality I tried several times after I applied my solution, there's no losing connection. Sometimes, having without usb tester is helpful. In my experience, which is not exactly clear tho, powerbank is less stable than power adapter. RPi 4 8GB is recommended 3A. for input. I tested with 3A adapter for RPi, 2A for powered usb hub.

I'm using wireless usb adapter for sending pose data and it takes 0.45A, 5V, which means I turned off RPi's wifi chip. And I disabled fisheye camera streaming, too(maybe you already tried).

Construkted-Reality commented 2 years ago

@HanDaSeul I just tried two different 3A power adapters (one was from an LG phone, and another from a Pixel 3 phone) and have experienced the same loss of connection. I have disabled the camera stream in the python script.

HanDaSeul commented 2 years ago

@Construkted-Reality Then, probably only I can say is use wireless usb wifi adapter with powered usb hub. Don't let other usb devices take power from RPi. Let RPi take less power as you can like setting RPi's wifi chip off. And just in case keep T265 cool. In my case as you see my screenshot, RPi gathers pose data and sends. My laptop receives that pose data, so I can see the t265's movement almost without lag through rviz.

Wait...it looks like the problem is not only the power, but also RPi's wifi chip performance...I didn't test using RPi's wifi chip.

Construkted-Reality commented 2 years ago

I don't think cooing the T265 has anything to do with it since I've had the T265 work perfectly on a Pi3, and it never disconnects (without any cooling). I'll do some tests with the wifi module disabled to see if that makes any difference.

zhouzhiwen2000 commented 2 years ago

@Construkted-Reality Then, probably only I can say is use wireless usb wifi adapter with powered usb hub. Don't let other usb devices take power from RPi. Let RPi take less power as you can like setting RPi's wifi chip off. And just in case keep T265 cool. In my case as you see my screenshot, RPi gathers pose data and sends. My laptop receives that pose data, so I can see the t265's movement almost without lag through rviz.

Wait...it looks like the problem is not only the power, but also RPi's wifi chip performance...I didn't test using RPi's wifi chip.

Try putting more pressure on the RPI USB port while testing, eg. copying a large file from an SSD. It seems that the problem occurs more often when the bus is busy. The SSD can be powered externally to avoid extra power consumption.

zhouzhiwen2000 commented 2 years ago

@Construkted-Reality Then, probably only I can say is use wireless usb wifi adapter with powered usb hub. Don't let other usb devices take power from RPi. Let RPi take less power as you can like setting RPi's wifi chip off. And just in case keep T265 cool. In my case as you see my screenshot, RPi gathers pose data and sends. My laptop receives that pose data, so I can see the t265's movement almost without lag through rviz. Wait...it looks like the problem is not only the power, but also RPi's wifi chip performance...I didn't test using RPi's wifi chip.

Try putting more pressure on the RPI USB port while testing, eg. copying a large file from an SSD. It seems that the problem occurs more often when the bus is busy. The SSD can be powered externally to avoid extra power consumption.

I tested using T265 and a USB camera, with the USB cam externally powered, and the problem is still around.

zhouzhiwen2000 commented 2 years ago

For those using T265, the ASUS tinker board seems to be a good choice. It's tested and no such problem is observed. Its price is also similar to RPI4.

DevepNoName commented 2 years ago

If you guys seach here about T265, that camera doesnt work properly in Raspberry 4 / Nvidia Jetson Nano, lots of people complaining about this (inluding myself), and Intel will not correct this (just shutdown after a while, and have to take out the cable and reconnect again). But works in Raspberry 3(not tested by myself), some people confirmed this.

DevepNoName commented 2 years ago

https://github.com/IntelRealSense/librealsense/issues/8779

giangthewalkingman commented 1 month ago

@DevepNoName If you guys seach here about T265, that camera doesnt work properly in Raspberry 4 / Nvidia Jetson Nano, lots of people complaining about this (inluding myself), and Intel will not correct this (just shutdown after a while, and have to take out the cable and reconnect again). But works in Raspberry 3(not tested by myself), some people confirmed this.

no, I 'm facing the similar problem in raspberry pi 3b+

giangthewalkingman commented 1 month ago

@silverjoda After plugging in and out about 200 times on the Rpi, I found out that you can use Uhubctl to automatically power cycle all the USB ports after boot, so that the t265 is recognized all the time.

did u mean that using uhubctl to manage power so that t265 will be connected all the time without odom losing?

silverjoda commented 1 month ago

The odom dropping out is one issue. The RS not being recognized after reboot (if I remember correctly) is another issue, one that is fixed using ububctl.

On Wed, 15 May 2024 at 04:45, Truong Giang Nguyen @.***> wrote:

After plugging in and out about 200 times on the Rpi, I found out that you can use Uhubctl to automatically power cycle all the USB ports after boot, so that the t265 is recognized all the time.

did u mean that using uhubctl to manage power so that t265 will be connected all the time without odom losing?

— Reply to this email directly, view it on GitHub https://github.com/IntelRealSense/librealsense/issues/6362#issuecomment-2111477926, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFAMSECFYN4JSFFBFINZLB3ZCLD3DAVCNFSM4M3CLYZ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJRGE2DONZZGI3A . You are receiving this because you were mentioned.Message ID: @.***>

-- Ing. Teymur Azayev PhD +420 725767494