IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.61k stars 4.83k forks source link

False pose when testing rs-motion with D435i #4915

Closed MingheCao closed 4 years ago

MingheCao commented 5 years ago

Required Info
Camera Model D435i
Firmware Version 05.11.11.100
Operating System & Version Ubuntu 16.04
Kernel Version (Linux Only) 4.15.0-62
Platform PC
SDK Version 2.25.0
Language
Segment

Issue Description

Hi, I got some pose drifting issues when testing VINS using D435i, so I checked the imu streams with the rs-motion module.

At the beginning of the application, the imu model rotates randomly, even if the camera was hold still.

IMB_zzYq1p

It usually takes several seconds that the model finally become stable. But the pose seems not right. - The 3D model will not reflect the yaw, and the rolling is presented oppositely as follows.

IMB_qOUe5A

Does anyone had the same issue, and how should I solve it? Appreciated!

ev-mp commented 5 years ago

@MingheCao hello, The failure to track left-right motion, as observed in the second video, is a clear indication of no valid gyro inputs. Can you check whether the Gyro stream is available when selected in viewer, and whether the generated data reflects actual motion ? Rotating the device left-right (yaw) 90 deg. at steady speed in one second should generate ~1.5 rad/sec angular velocity about Y axis

MingheCao commented 5 years ago

@ev-mp Thank you for your quick reply. I checked the Gyro stream and I think it can provide a reasonable output as I do the left-right motion.

I also bundled the camera with a 3dm-gx5-25 IMU and processed the data using ROS imu_complementry_filter package. The 3dm imu gives a satisfactory result as follows, where the x-axis is time and the y-axis is rotation angle.

mjclaacgnchohiac

Oppositely, the outcome of camera imu is messy.

hpiflnmnhmmjnamf

I highly doubt it is the imu streams problem. Do I have an approach to find hard evidence?

ev-mp commented 5 years ago

@MingheCao hello, Can you confirm that when running realsense-viewer you do observe the gyro generating 'reasonable' data ? As it seem not consistent with the video clips provided.

Also, when running with viewer please check whether the timestamps of Gyro and Accelerometer are in the same domain and roughly identical, as this could be a timestamp-related issue that affects ROS IMU topic. To get HW/Global Time timestamps for accel and gyro the kernel patch must be applied image

MingheCao commented 5 years ago

I am using v2.25.0 in accordance with the ROS wrapper. But I think this problem has been solved in the lastest release v2.28.1. I only encounter this problem once out of many attempts in v2.28.1.

RealSenseCustomerSupport commented 5 years ago

@MingheCao is there anything further needed here? I see that you're getting better data via the latest release.

for-no-one commented 5 years ago

@ev-mp @MingheCao @RealSenseCustomerSupport Hello, I encountered the same issue in rs-motion example with the latest library and firmware (using Ubuntu 18.05, kernel 4.15). Approximately every 10th time I start rs-motion example, the camera rotates like crazy in the application even though the actual camera is stationary. After a few hours of debugging I think I found the reason for this behavior:

The example assumes that the first gyroscope frame is received before the first accelerometer frame. Both process_gyro and process_accel handle their first frame differently from following frames. They share a common flag, which is set to False in process_accel when it receives its first frame. Because of that, if the first acceleration frame is received before gyroscope frame, process_gyro never gets to execute the block that should be executed on the first iteration which results in a huge timestamp for gyroscope data on next iteration which sends the camera rotating like crazy.

LuckierDodge commented 4 years ago

I'm seeing a similiar issue consistently whenever I run rs-motion, realsense-viewer, or the ROS-wrapper on Ubuntu 18.04. Angular Y and Z read as +56.765 and -22.888, without any variance. Angular X, as well all 3 linear velocities, behave normally. The issue isn't present on my windows machine.

Calibrating and updating firmware had no effect.

RealSenseCustomerSupport commented 4 years ago

Hi @LuckierDodge and @MingheCao

Taking for a moment the setups and camera to be the same and just the operating system is different in case of seeing this happening in Ubuntu 18.05, kernel 4.15 but not happening in Windows, leads me to believe that the issue could be operating system related. I've updated my D435i to latest FW on Windows platform and have not been able to reproduce the issue which is in accordance with @LuckierDodge assessment.

Has this been tried with latest Librealsense release along with latest FW release and Ubuntu 16?

LuckierDodge commented 4 years ago

@RealSenseCustomerSupport I was seeing the problem with the latest apt release on Ubuntu 18.04, was able to at least temporarily fix it by building v2.28.1 from source (trying to install an old release from apt proved problematic). FW was up to date for both. Haven't had the chance to try it with Ubuntu 16.04 though.

RealSenseCustomerSupport commented 4 years ago

Hi Minghe,

We are unable to reproduce this issue on latest SDK and Firmware. Are you still seeing this?

RealSenseCustomerSupport commented 4 years ago

Will you be needing further assistance with this? Otherwise we can close this issue.

RealSenseCustomerSupport commented 4 years ago

Hi @MingheCao

Will you be needing further assistance with this? Otherwise we can close this issue.

If we don’t hear from you in 7 days, this issue will be closed.

prparsons commented 3 years ago

FYI, this issue is still occuring. I just downloaded the win10 sdk (Intel.RealSense.SDK-WIN10-2.38.1.2223 ) and get this when running the rs-motion it installed. The realsense viewer is reporting data correctly, so seems it would be an application bug, not hardware or library issue (I'm not worried about, but figured I'd mention it)

RealSenseSupport commented 3 years ago

@prparsons There's one PR https://github.com/IntelRealSense/librealsense/pull/7500 just merged to development branch to fix bugs in rs-motion. Could you please try if that fix your issue? Thanks!