calibration issue (IMU to ouster)

[naderabdelaziz]

Hello, Thank you for making this dataset available to researchers. I have spent a good amount of time dealing with your dataset, albeit not through the API platform for most tasks.

My ultimate goal is to transform a SLAM solution from the camera frame (flir_bfc) to the ENU navigation frame. The logic of this transformation is: RT_flirbfc_ENU = RT_sbgekinox_ENU RT_ouster_sbgekinox RT_flir_bfc_ouster For this transformation, the information used is the extrinsic calibration parameters you provide, and the GPS/INS data as found in sbgekinox_bcc_navposvel and sbgekinox_bcc_ekfeuler, respectively.

I have noticed some problems that I could not figure out:

1. IMU axes configuration: you included a plot in your published paper showing the data collection platform. However, you did not plot anything about how the IMU is mounted (i.e. the body frame: is x forward or y forward?). So, could you please clarify this?
2. I have tried to figure this out on my own by inspecting the values you have in the dataset, however, I came across some contradictions that I cannot resolve: a. If you inspect the values of the acceleration in the z-direction of the IMU data ("sbgekinox_bcc_imudata"), you will find the values range around -9.81. The minus sign indicates the IMU is mounted such that the Z is down, and I would guess that in such a case, the X is the forward axis based on the type of IMU you are using. b. The problem with the previous hypothesis is that if you inspect the extrinsic calibration matrix between the IMU and the ouster as in "sbgekinox_bcc-ouster64_bfc", you will see that the rotation is almost an identity matrix. This indicates that the axes of the IMU coincide with those of the ouster, meaning the x is forward, y is left, and z is up. This is contradictory to the readings of the IMU as mentioned in point 1 c. This also raises another important question, are the roll, pitch, and yaw values in reference to the ENU navigation frame or the NED? d. How is the yaw angle is measured (from the east or from the north)? e. These will affect how the transformation is executed from the body frame to the ENU/NED frame

The below information refers to what I have previously mentioned:

A sample of the IMU data (see the -9.998819 for acc_z value) 4037401722.000000 1023.000000 1.210357 -0.388774 -9.998819 -0.006661 -0.026723 0.004820 41.894199 1.210357 -0.388774 -9.998819 -0.006661 -0.026723 0.004820

The transformation from the IMU to the ouster (rotation is almost identity) 0.999658 -0.007231 -0.025142 -3.907172 0.007107 0.999962 -0.005035 -0.011117 0.025177 0.004855 0.999671 -0.322196 0.000000 0.000000 0.000000 1.000000

Sample image to show you what I mean about the IMU axes (source: KITTI dataset)

[XavierOuimet-Leddartech]

Hi Nader,

Thanks for your question!

1. The SBGEkinox sensor is both an IMU and GPS. The referential is Z up, X forward, Y left

2. a. An acceleration of -9.81 means that the acceleration is pointing down towards the earth, meaning Z up. b. Having Z up, this is in line with the Lidar referential, hence the identity rotation matrix. c. The referential of the roll, pitch and yaw is ENU. The is from the fused INS data. d. Yaw angle should be from north

Here is a small code sample that could help extract the transformation as a 4x4 matrix transform from world to ego:

from pioneer.common import linalg for i in range(len(pf['sbgekinox_bcc_navposvel'])): nav = pf['sbgekinox_bcc_navposvel'][i].raw euler = pf['sbgekinox_bcc_ekfeuler'][i].raw tf = linalg.tf_from_poseENU(linalg.imu_to_utm(nav, euler), dtype = np.float64)

Also some utilities might be used from pioneer.common: https://github.com/leddartech/pioneer.common/blob/master/pioneer/common/imu_utils.py

[naderabdelaziz]

Hi XavierOuimet,

Your quick response is much appreciated! Thank you for the clarifications. I would not fully agree with some of the points you mentioned, especially 2.a. I am still running tests on my end based on the lines of code you thankfully provided me as well as codes that I developed myself that are independent of your Python API.

However, in the meantime, could you write lines of code to continue the transformation you have suggested to take me to the flir_bfc camera frame? The sequence of this should be:

NB: I am using different synonymous terms to ensure we are on the same page

World (WGS84) --> Navigation frame (ENU) --> Body frame (IMU) (NB: I think you are referring to this as ego pose) --> ouster --> flir_bfc

I will verify some experiments I am running and get back to you with a detailed response about what I think in light of your reply, but I need to complete testing first.

Once again, Thanks! I look forward to your response!

Best regards, Nader Abdelaziz.

[XavierOuimet-Leddartech]

Hi Nader!

The Z axis from the IMU is pointing up (positive), since earth's gravity is pointing down. This makes for a negative value of acceleration along the Z axis.

We suggest using the built-in das api methods to transform data from one referential to another. Most sample do have a get method in which you can specify the target referential.

For lidar points, the get_point_cloud method has a "referential" argument in which you can specify the target referential: https://github.com/leddartech/pioneer.das.api/blob/1f18e0fc40f1ea2963fadee035b96ef72fdd3baf/pioneer/das/api/samples/point_cloud.py#L15