search

ankitdhall / lidar_camera_calibration

ROS package to find a rigid-body transformation between a LiDAR and a camera for "LiDAR-Camera Calibration using 3D-3D Point correspondences"
http://arxiv.org/abs/1705.09785
GNU General Public License v3.0
1.49k stars 460 forks source link

flipped camera points #22

Closed ghost closed 6 years ago

ghost commented 6 years ago

We have a testing setup as shown in the image and got some results from it. I evaluated the result using the matlab code provided. But seems the result of camera points are flipped. Our velodyne in the car is pointing downwards around 6 degrees, camera is pointing down around 15 degrees. I subscribed to the already rectified image topic. Not sure what could be the reason for this result? averaged transformation T = [ 0.907442 -0.377166 0.185188 -0.656001 -0.372771 -0.926022 -0.0593794 -0.212374 0.193884 -0.0151491 -0.980908 3.65862 0 0 0 1 ]; evaluation setup

karnikram commented 6 years ago

Are the aruco markers oriented properly? Its Z axis should be along edge s2 and X axis should be along edge s1, as shown in the README.

ghost commented 6 years ago

thanks for pointing out, yep the aruco tag is up side down. (x-axis along the breadth (s2) and z-axis along the length (s1)).. We now have a new result and seems matlab evaluation is good. evaluate

One more question is what is the unit of translation? our new matrix is T = [ 0.991569 -0.0102226 0.129178 -0.242547; -0.00371206 0.994234 0.107172 -0.638973; -0.129529 -0.106748 0.985813 -0.00492584; 0 0 0 1]; but our setup doesn't look like 0.63m in y axis..

ghost commented 6 years ago

I'm trying to project Velodyne pointcloud to the image. So I got pointcloud xyz from lidar, add 1 at the end. Then the points in camera frames should be T xzy' From camera frame to image, x = focal_length X / Z + principal_point But after this process, velodyne points are not showing in the image... Just wondering it's the problem of calibration or the way I project it? Really appreciate your help!

karnikram commented 6 years ago

The quantities are in meters. -0.63m signifies your lidar is 63 metres above the camera.

Does T account for the initial transformation too? If both are forward facing, T needs to account for the initial (pi/2, -pi/2, 0) rotation and not just the estimated calibration value.

ghost commented 6 years ago

yep, So I should rotate the velodyne points by initial rotation first, and then transformed by T. But the results with several trials are not aligned really well. Not sure what's the issue..

villanuevab commented 6 years ago

Could the MATLAB script output be considered re-projection error? Similar to re-projection error calculated during camera calibration?

A few question about the plot:

karnikram commented 6 years ago

It outputs the RMSE error, it is not the re-projection error. It is a plot of the 3D points after transforming all of them to the camera's frame. The units are in meters. It is not a 2D plot even though it appears so.

On Mon, May 7, 2018 at 11:51 PM Blanca V notifications@github.com wrote:

Could the MATLAB script output be considered re-projection error? Similar to re-projection error calculated during camera calibration?

A few question about the plot:

  • What are the units of x/y axes in the plot? Meters?
  • Is the coordinate frame depicted in the plot the camera frame or the image frame?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ankitdhall/lidar_camera_calibration/issues/22#issuecomment-387156309, or mute the thread https://github.com/notifications/unsubscribe-auth/AMETK8MsUGzYhKfHdtdv9AJotDZa3Nr8ks5twJCKgaJpZM4QUVrN .

villanuevab commented 6 years ago

I see. So, for example, if I got these results: 

---
After 100 iterations
--------------------------------------------------------------------
Average translation is:
0.628969
0.686662
0.768073
Average rotation is:
   0.999705  0.00735936   0.0231482
-0.00219281    0.976449   -0.215735
 -0.0241907    0.215621    0.976177
Average transformation is: 
   0.999705  0.00735936   0.0231482    0.628969
-0.00219281    0.976449   -0.215735    0.686662
 -0.0241907    0.215621    0.976177    0.768073
          0           0           0           1
Final rotation is:
  0.0239442    -0.99968 -0.00815538
  -0.215737  0.00279854   -0.976447
   0.976158   0.0251397   -0.215601
Final ypr is:
  1.68133
  -1.7896
-0.116079
Average RMSE is: 0.0326677
RMSE on average transformation is: 0.0472742

Then using the MATLAB script should show roughly 4.7cm of difference between the camera points (red) and lidar points (blue) when averaged -- it might be a little off since we don't see the difference in z. Is this correct?

karnikram commented 6 years ago

Yeah. It'll have the same RMSE.

On Wed, May 9, 2018 at 4:50 AM Blanca V notifications@github.com wrote:

I see. So, for example, if I got these results:

After 100 iterations

Average translation is: 0.628969 0.686662 0.768073 Average rotation is: 0.999705 0.00735936 0.0231482 -0.00219281 0.976449 -0.215735 -0.0241907 0.215621 0.976177 Average transformation is: 0.999705 0.00735936 0.0231482 0.628969 -0.00219281 0.976449 -0.215735 0.686662 -0.0241907 0.215621 0.976177 0.768073 0 0 0 1 Final rotation is: 0.0239442 -0.99968 -0.00815538 -0.215737 0.00279854 -0.976447 0.976158 0.0251397 -0.215601 Final ypr is: 1.68133 -1.7896 -0.116079 Average RMSE is: 0.0326677 RMSE on average transformation is: 0.0472742

Then using the MATLAB script should show roughly 4.7cm of difference between the camera points (red) and lidar points (blue) -- it might be a little off since we don't see the difference in z. Is this correct?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ankitdhall/lidar_camera_calibration/issues/22#issuecomment-387572383, or mute the thread https://github.com/notifications/unsubscribe-auth/AMETK4SaIi4H4P-giztyoxlZjIQ46MSvks5twigxgaJpZM4QUVrN .

oaix commented 5 years ago

I see. So, for example, if I got these results:

---
After 100 iterations
--------------------------------------------------------------------
Average translation is:
0.628969
0.686662
0.768073
Average rotation is:
   0.999705  0.00735936   0.0231482
-0.00219281    0.976449   -0.215735
 -0.0241907    0.215621    0.976177
Average transformation is: 
   0.999705  0.00735936   0.0231482    0.628969
-0.00219281    0.976449   -0.215735    0.686662
 -0.0241907    0.215621    0.976177    0.768073
          0           0           0           1
Final rotation is:
  0.0239442    -0.99968 -0.00815538
  -0.215737  0.00279854   -0.976447
   0.976158   0.0251397   -0.215601
Final ypr is:
  1.68133
  -1.7896
-0.116079
Average RMSE is: 0.0326677
RMSE on average transformation is: 0.0472742

Then using the MATLAB script should show roughly 4.7cm of difference between the camera points (red) and lidar points (blue) when averaged -- it might be a little off since we don't see the difference in z. Is this correct?

@villanuevab I have no idea about the rotation matrix. Which one should I use, final rotiaon or average rotation? What is the differnence between the two matrix?