search

ankitdhall / lidar_camera_calibration

ROS package to find a rigid-body transformation between a LiDAR and a camera for "LiDAR-Camera Calibration using 3D-3D Point correspondences"
http://arxiv.org/abs/1705.09785
GNU General Public License v3.0
1.52k stars 461 forks source link

monocular camera image with Lidar Points #112

Open SinghiDivyanshu opened 4 years ago

SinghiDivyanshu commented 4 years ago

hi,

what is the next step after finding the rigid body transformation using Velodyne LIDAR and monocular camera. my RMSE value is 0.1636 as shown in the image.

so how to find the image with LIDAR point cloud on it. output_camera_calibration

SiddhantNadkarni commented 4 years ago

Hi @SinghiDivyanshu, once you have the rigid body 4x4 transformation matrix you'll have to perform the following steps:

  1. Transform the point cloud captured from LiDAR to Camera Co-ordinate system by using the 4x4 Rigid Body Transformation matrix obtained from this package.
  2. Project the Points from Camera Co-ordinate system to image plane by using 3x4 Camera Instrinsic / Projection Matrix which contains camera info such as focal lengths etc.

Let me know if you have doubts.

AnthonyLadewig commented 3 years ago

Hi @SiddhantNadkarni ,

How you do transform the point cloud to the camera coordinate system and then project it? I don't know what packages or code to use.

dkhanna511 commented 3 years ago

Can you please mention what changes you had to make in the launch and lidar_camera_calibration.yaml to get it running, my topic names are different and I'm not able to understand which change has to be made where.

AnthonyLadewig commented 3 years ago

@dkhanna511 In the launch file, you have to uncomment the code under aruco mapping and then change the remap topic.

remap from="image_raw" to="zed/left/image"

Change the zed/left/image to your camera's topic.

In the yaml file, do the same.

To find your camera's topics, run the launch for your camera and do 'rostopic list' in the terminal.

dkhanna511 commented 3 years ago

@AnthonyLadewig
Hi, thanks for the response. I did these changes, Now I'm facing an openCV issue. that is mentioned in #131. Do you have idea about how to solve that? Have you encountered this issue?

AnthonyLadewig commented 3 years ago

@dkhanna511 Unfortunately I have not had this problem and do not know how to fix it. Sorry

dkhanna511 commented 3 years ago

@dkhanna511 Unfortunately I have not had this problem and do not know how to fix it. Sorry

@AnthonyLadewig I ran the same process on another laptpo and it works. Apparently there are some OpenCV conflict issues. After running the find_transform.launch file, only mono8 window is opening. I have not created the setup as of now but I want to know if the polygon window and point cloud window pops up when there is no marker at first.

AnthonyLadewig commented 3 years ago

@dkhanna511 from my experience, the polygon/point cloud window doesn't come up if the lidar isn't running.

dkhanna511 commented 3 years ago

@dkhanna511 from my experience, the polygon/point cloud window doesn't come up if the lidar isn't running.

@AnthonyLadewig The lidar is running. I can see the point cloud fees on RVIZ

AnthonyLadewig commented 3 years ago

@dkhanna511 Im not sure what the problem might be then. Is the calibration software actually subscribing to the lidar topic? What lidar are you using btw?

dkhanna511 commented 3 years ago

@AnthonyLadewig I was using VLP16. I had to install everything from scratch and now all the 3 windows come up.