search

heethesh / lidar_camera_calibration

Light-weight camera LiDAR calibration package for ROS using OpenCV and PCL (PnP + LM optimization)
BSD 3-Clause "New" or "Revised" License
528 stars 117 forks source link

Correspondence GUI takes very long to show up #18

Open gonultasbu opened 3 years ago

gonultasbu commented 3 years ago

Hello, I have an issue with the point selection GUI. After pressing enter, it takes a long time time to show up, is this a performance limitation or is there a trigger for it within the source code? I looked for a trigger within the code for a bit but haven't been able to find one.

Thank you for this great repository.

heethesh commented 3 years ago

If the number of points in your point cloud is too large, it might be a performance issue as Matplotlib does not handle this well. For this reason, I have added a limit on the field-of-view to display - if this also does not help, consider downsampling your point cloud (which probably would increase some error in calibration). Please check the number of points to confirm if this is the issue. Unfortunately I don't have a solution for this now, working on an update to switch to Open3D or Rviz soon.

gonultasbu commented 3 years ago

The number of points is pretty similar to the number of points in the LiDAR calibration since I'm also working with VLP-16. I will put further work on this issue and will let you know if I come up with anything interesting.

gonultasbu commented 3 years ago

I also have another question related to the correspondences:

after solving PnPRANSAC, do we get a transform from the velodyne frame of reference to the world frame of reference? According to the OpenCV documentation, this is the case, however we are publishing the transform as the world -> velodyne using the static transform. Since we use this static transform to transform velodyne point cloud, which is in the velodyne frame of reference. This actually transforms the points from velodyne frame of reference to the world frame of reference, although the naming convention of the static transform indicates the opposite.

heethesh commented 3 years ago

There is no world frame during the calibration. You are estimating the camera -> velodyne transformation. The world frame is just a dummy base frame used to transform the point cloud and visualize it in Rviz. Make sure to select the fixed frame as world in Rviz (edit: I'm actually not sure what the fixed frame was when I made the video, try switching it velodyne if it does not work).

gonultasbu commented 3 years ago

I see, what I was saying was the transformation we get after solving PnPRANSAC is the T matrix that does the velodyne -> world transform. world being the dummy base frame indeed.

But this transformation is published as:

X Y Z Y P R args="0 0 0 0 0 0 world velodyne" />

this implies the transform is as follows: world -> velodyne.

Since the numeric operations done follow the correct convention, the result is correct. It's the written semantics that might be misleading.