How to use average translation and YPR angles with ROS from LIDAR and Camera data fusion

[jediofgever]

Hi, I run the package, and got results as shown below,

I would like to visualize Lidar point clouds in Zed camera frame. So I do a static coordinate transformation with this command

rosrun tf static_transform_publisher -0.09122 -0.04545 -0.0432 1.48563 -1.5026 0.13215 zed_left_camera velodyne 100

the first three parameters are x y z for translation and the last three parameters are for yaw pitch roll angles.

However result look non relevant as shown below

What I am doing wrong ? Is there another way to use transformation matrix(4X4) in ROS below are my files and rosbag zed_left_uurmi.ini config_file.txt

marker_coordinates.txt Rosbag : https://drive.google.com/open?id=1HKqOUeIacSARwwUcCV6CU83fjTWr3OwI

Update; My setup

[karnikram]

Hi,

What is the white colored plane in your visualization? That doesn't seem like the point cloud from your zed.

You could alternatively run stereo_image_proc for obtaining the zed's point cloud and then fuse them together in rviz like you've done.

Another way of visualizing the results is described in #19 and #20

[jediofgever]

Hi, The white plane is depth cloud from left camera, the point cloud that I am going to receive from stereo_image_proc will be at the same frame with the one on white plane, so I don’t think it will solve my problem. However I will give it a try.

[jediofgever]

There is the section in your code , pointcloud_fusion. But it is designed for fusing 2 camera point clouds. However I just want to fuse Lıdar and zed camera point clouds. Can you tell me how can I use that part of code for Lidar and one zed camera point cloud fusion ?

[karnikram]

If you used the ZED SDK to obtain that depth cloud then that might be the problem since they use a different axes convention.

You can modify pointcloud_fusion for your case by directly giving the final transform, provided you have the respective pcd files.

[jediofgever]

Right, I use zed sdk. Last question, I am not sure if the problem is the way I fuse data or the calibration parameters are not correct ?, Do the calibration parameters my program found seems reasonable ?

[karnikram]

Your calibration parameters do seem correct to me.

The way you're verifying them in rviz is also correct, but like I said zed uses a different axes convention and not the standard convention. According to their convention x points forward, y towards the left, and z upward, which is the same as the velodyne's. The two initial rotations (about x and y) are not needed in that case.

But this package uses the standard ROS convention for the camera axes, which needs the initial rotations, and hence there's a mismatch in your output. I suggest you use stereo_image_proc and visualize it again.

[jediofgever]

I tried to use stereo_image_proc, however I stuck on getting topics not synchronized error. Can you share your launch file that you use. such as roslaunch stereo_image_proc your.launch Thank you

[karnikram]

I guess you're missing camera_info messages. I do not have a launch file for stereo_image_proc, I just run it from the command line with the __ns argument.

[jediofgever]

Thank you for all the help. ! Got it working

[karnikram]

Can you post your solution here for reference?

[jediofgever]

Here is the command I pass to get over of it ; rosrun stereo_image_proc stereo_image_proc __ns:=zed _approximate_sync:=True