search

stereolabs / zed-ros-wrapper

ROS wrapper for the ZED SDK
https://www.stereolabs.com/docs/ros/
MIT License
448 stars 391 forks source link

Different coordinate system between camera and IMU #809

Closed YuxuanSnow closed 2 years ago

YuxuanSnow commented 2 years ago

Preliminary Checks

Description

Hi,

i wonder if the coordinate system of cameras and IMU are different? I tried to calibrate the extrinsic using Kalibr and Basalt. However, the optimization always converges and I think it's due to the poor initialization. I used the identity matrix as the initial Rotation matrix, but it not correct. Could you please tell me the exact coordinate system of cameras and IMU? (better with an intuitive example)

Thanks and with best regards

Steps to Reproduce

  1. Use kalibr to calibrate the extrinsic between cameras and IMU

Expected Result

The extrinsic between cameras and IMU

Actual Result

Calibration diverges

ZED Camera model

ZED Mini

Environment

OS: ubuntu 18.04

Anything else?

No response

Myzhar commented 2 years ago

Hi @YuxuanSnow Camera and IMU have the same coordinate system. The ZED ROS Wrapper following the ROS convention illustrated in the REP105: X forward, Y left, Z up

YuxuanSnow commented 2 years ago

Hi @Myzhar Thank you for your answer. I think then the problem should be that the Kalibr assumes the Z-axis is forward. I will try again.

mandelyoni commented 1 year ago

Dear Sir,

I'm using Zed2i's sensors (left camera and IMU) with an external monocular VIO pipeline. The extrinsics I receive from the ROS topic /left_cam_imu_transformation and from Kalibr are very different.

The extrinsics from /left_cam_imu_transformation is: rostopic echo /zed2i/zed_node/left_cam_imu_transform translation: x: -0.0020000000949949026 y: -0.023000003769993782 z: 0.0002200000308221206 rotation: x: -0.0010500885546207428 y: -0.0 z: -0.00393286207690835 w: 0.9999917149543762 which is equivalent to - [0.999969, 0.007864, -0.000099, -0.002000, -0.007864, 0.999967, 0.002121, -0.023000, 0.000116, -0.002120, 0.999998, 0.000220, 0, 0, 0, 1]

While the extrinsics from Kalibr (https://support.stereolabs.com/hc/en-us/articles/360012749113-How-can-I-use-Kalibr-with-the-ZED-Mini-camera-in-ROS-) are : cam0: T_cam_imu:

[-0.004248117180815025, -0.9998625130142822, -0.016028367639358054, 8.41951564249623e-05] [0.001063559295260974, 0.0160239856188632, -0.9998710420481793, -1.708597954171419e-06] [0.9999904111250486, -0.004264616471714999, 0.0009953413010867496, 3.811715513237331e-06] [0.0, 0.0, 0.0, 1.0] cam_overlaps: [1] camera_model: pinhole distortion_coeffs: [0.0006582543326556634, -0.001338594740114625, 0.0003683086103016461, 0.0016571045464767074] distortion_model: radtan intrinsics: [662.3788576876452, 662.6400636208534, 654.9267980662693, 366.0819275708851] resolution: [1280, 720] rostopic: /zed/zed_node/left/image_rect_color timeshift_cam_imu: 0.10015660064007245 (also strange) In addition, while the camera is horizontal and facing forward, the output of /zed2i/zed_node/imu/data_raw is: header: seq: 17969 stamp: secs: 1681195935 nsecs: 453395218 frame_id: "zed2i_imu_link" orientation: x: 0.0 y: 0.0 z: 0.0 w: 0.0 orientation_covariance: [-1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] angular_velocity: x: -0.058271501732520346 y: -0.1356413144708724 z: 0.06934011135360624 angular_velocity_covariance: [6.595000957109997e-10, 0.0, 0.0, 0.0, 1.9613346042712762e-09, 0.0, 0.0, -0.0, 7.858832729846918e-10] linear_acceleration: x: 1.5133049488067627 y: 0.3734425902366638 z: 9.75547981262207 linear_acceleration_covariance: [0.007840245962142944, 0.0, 0.0, 0.0, 0.008192301727831364, 0.0, 0.0, -0.0, 0.007778662722557783]

You can see the z axis of linear_acceleration is pointing up. According to /left_cam_imu_transform it should be pointing forward (like the camera).

Please advise.

Thank you, Yoni

Steps to Reproduce rostopic echo /zed2i/zed_node/left_cam_imu_transform Follow https://support.stereolabs.com/hc/en-us/articles/360012749113-How-can-I-use-Kalibr-with-the-ZED-Mini-camera-in-ROS- rostopic echo /zed2i/zed_node/imu/data_raw Expected Result Transformation should look similar

Actual Result Transformation not similar

ZED Camera model ZED2i

Environment ROS Noetic on Ubuntu 20.0 on an intel chip (Lenovo Thinkpad)

Myzhar commented 1 year ago

@mandelyoni please do not cross-post (https://github.com/stereolabs/zed-ros-wrapper/issues/335#issuecomment-1514162061) and do not reply to closed issues, it's preferred to open a new issue and cite the closed one if you need it.