Error while calibrating camera-lidar

[poornimajd]

Great work @ankitdhall and team! I tried using this implementation with velodyne-32C and Zed camera.This is my setup- I have changed the topic names in the config file ,the marker coordinates ,and uncommented aruco-mapping in launch file. I am using live sensor feed from lidar and camera,and when I played in rviz,it displays both the image and point cloud properly.(shown below) This indicates that there is time sync between camera and lidar. But eachtime I run the command - roslaunch lidar_camera_calibration find_transform.launch

I get the following error Only Mono8 window displays,the lidar (pointcloud window) is not displayed. I checked the topics getting published, the lidar_camera_calibration_rt is getting published but when I echo it gives the output as - are your messages being built? My question is when ever I run the command why does the process die and throw that error as shown in the picture and the lidar window also does not diaplay. I checked out similar issues but I was unable to get a solution. Any suggestion is greatly appreciated.

[yulan0215]

I have the same problem, do you solve it now? Thanks

[poornimajd]

No,not yet.

[dkhanna511]

I have the same problem. I built the packages required. And launched the point grey camera and VLP 16 Lidar. However, I haven't really created the setup for now and I ran the find_transform.launch file and I'm having this issue. Can you let me know what result is expected if we connect everything without the setup and launch the file?

[miriamrebekah]

@ankitdhall, I have encountered the same issue. point_cloud seems to go out of scope inside intensityByRangeDiff(point_cloud,config). After looping several times through the pointcloud in intensityByRangeDiff(), the function throws a std::bad_alloc.

in find_velodyne_points.cpp (the extra couts added just to double check where I am)

qlidarToCamera = Eigen::AngleAxisd(config.initialRot[2], Eigen::Vector3d::UnitZ())
        *Eigen::AngleAxisd(config.initialRot[1], Eigen::Vector3d::UnitY())
        *Eigen::AngleAxisd(config.initialRot[0], Eigen::Vector3d::UnitX());

lidarToCamera = qlidarToCamera.matrix();

std:: cout << "\n\nInitial Rot" << lidarToCamera << "\n";
std:: cout << "\n\nPoint Cloud" << point_cloud << "\n";
std:: cout << "\n\nGoing into intensityByRangeDiff" << "\n";
point_cloud = intensityByRangeDiff(point_cloud, config);

config_file.txt:

2048 1080
-2.5 2.5
-4.0 4.0
0.0 1.7
0.05
2
0
1280.360906 0.0 1063.497160 0.0
0.0 1216.015996 453.016297 0.0
0.0 0.0 1.0 0.0
100
1.57 -1.57 0
0

command line output:

[ INFO] [1617968479.957185770]: Calibration file path: /home/simulator/catkin_ws/src/aruco_mapping/data/front_camera_dart.ini
[ INFO] [1617968479.957686996]: Number of markers: 2
[ INFO] [1617968479.957707574]: Marker Size: 0.252
[ INFO] [1617968479.957716174]: Type of space: plane
[ INFO] [1617968479.957724645]: ROI allowed: 0
[ INFO] [1617968479.957749894]: ROI x-coor: 21896
[ INFO] [1617968479.957760687]: ROI y-coor: 21896
[ INFO] [1617968479.957770377]: ROI width: 21896
[ INFO] [1617968479.957779263]: ROI height: -1968267360
[ INFO] [1617968479.958747080]: Calibration data loaded successfully
Size of the frame: 2048 x 1080
Limits: -2.5 to 2.5
Limits: -4 to 4
Limits: 0 to 1.7
Number of markers: 2
Intensity threshold (between 0.0 and 1.0): 0.05
useCameraInfo: 0
Projection matrix: 
[1280.361, 0, 1063.4972, 0;
 0, 1216.016, 453.0163, 0;
 0, 0, 1, 0]
MAX_ITERS: 100
Lidar_type: Velodyne
initial rotation: 1.57 -1.57 0
initial translation: 0 0 0
[ INFO] [1617968479.968735733]: Reading CameraInfo from configuration file
[ INFO] [1617968479.969293818]: Current velo topic is: 
[ INFO] [1617968479.969307117]: /vlp32c_roof_front/velodyne_points
[ INFO] [1617968480.354992175, 1614894028.400831280]: in big no cam callback
[ INFO] [1617968480.355044470, 1614894028.400831280]: Velodyne scan received at 1.61489e+09
[ INFO] [1617968480.355060500, 1614894028.400831280]: marker_6dof received at 1.61489e+09

Initial Rot 0.000796274    -0.999999 -0.000796274
           0  0.000796274           -1
           1  0.000796274  6.34053e-07

Point Cloudpoints[]: 47909
width: 47909
height: 1
is_dense: 1
sensor origin (xyz): [0, 0, 0] / orientation (xyzw): [0, 0, 0, 1]

Going into intensityByRangeDiff
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
[find_transform-2] process has died [pid 18211, exit code -6, cmd /home/simulator/catkin_ws/devel/lib/lidar_camera_calibration/find_transform __name:=find_transform __log:=/home/simulator/.ros/log/2abd29aa-9922-11eb-ad9b-48f17fd061c3/find_transform-2.log].
log file: /home/simulator/.ros/log/2abd29aa-9922-11eb-ad9b-48f17fd061c3/find_transform-2*.log

[ankitdhall]

@miriamrebekah thanks for the detailed log. I am not actively maintaining this repo; could you create a merge request that solves this issue that you encounter?

[cppchuff]

I met the issue when I use the rslidar-32, do you solve it? Many thanks.

[cppchuff]

https://github.com/ankitdhall/lidar_camera_calibration/blob/13d52954fa18ee3eef86272757555a28a2532c71/include/lidar_camera_calibration/PreprocessUtils.h#L200. change the num from 16 to 32. It will be work.