search

utiasASRL / pyboreas

Devkit for the Boreas autonomous driving dataset.
BSD 3-Clause "New" or "Revised" License
90 stars 9 forks source link

Coordinates of camera and lidar data recorded #34

Closed nithinme3 closed 1 year ago

nithinme3 commented 1 year ago

Hi Team, Thanks for the great dataset and repo!!

I have a doubt. I was trying the camera lidar projection tutorial code given in repo. It works perfectly well. But I was checking the camera coordinate converted x,y,z values of lidar points and x,y,z values of motion compensated lidar point clouds. I found some big difference. My doubt is, whether one coordinate of lidar is aligned with the forward direction of vehicle? similarly camera also? If it is so then there should be approaximately 0.64m distance difference between corresponding coordinates right? But am not able to match any of the camera lidar coordinate values in that respect.

Could you please help me where my understanding went wrong?

keenan-burnett commented 1 year ago

Hi @nithinme3,

The lidar is pointed at an approximately 45 degree angle away from "straight" The z-axis of the camera is pointing forwards.

Note that the camera and lidar are moving and that their measurements are not synchronous. Meaning, these measurements are occurring at different points in space and time. Because of this, when the vehicle is moving, the extrinsic transformation T_camera_lidar is going to change over time since several milliseconds have elapsed between then the lidar points were captured and the camera image was captured. During that time the vehicle has physically moved.

This can be understood most easily by looking at T_camera_lidar in our tutorial:

At the beginning of sequence boreas-2021-09-02-11-42 (t=1630597331.058472), the transformation is what you would expect from the calibration:

T_camera_lidar:
[[ 7.29301440e-01 -6.84192302e-01 -5.50358992e-04 -2.31580716e-02]
 [-1.29514464e-02 -1.30010849e-02 -9.99831602e-01 -2.29363626e-01]
 [ 6.84069930e-01  7.29185755e-01 -1.83429898e-02 -6.35368145e-01]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]

However, when the vehicle is moving, this offset will change over time. For example at (t=1630597590.0079348) we get

T_camera_lidar:
[[ 0.72906339 -0.68444466 -0.00144163 -0.02170728]
 [-0.01439054 -0.01322277 -0.99980902 -0.23317121]
 [ 0.68429488  0.72894489 -0.01948977 -1.02116481]
 [ 0.          0.          0.          1.        ]]

Let me know if this answers your question, or provide me with more details.