Using per-point stamps/timed point clouds optimally

[ojura]

Hello everyone,

I have a question. So, I have per-point data available for 3D lidars, which PointCloud2 does unfortunately not support. The approach I've taken to make some use of this information is to split points of each revolution into a number of bins (20-30?) according to their per-point stamps, make a PointCloud2 for each bin and stamp each bin according to the median timestamp.

Do you think there are better ways to do this? Perhaps create a new ROS message which would include per-point stamps, and construct a TimedPointCloud using information from that message? Do you think expanding cartographer_ros with something like that would be feasible? @SirVer I know discussions like this were had before, but couldn't find them.

[gaschler]

For your own experiments, just create some new ROS message format and in MsgConversion implement your own ToPointCloudWithIntensities in cartographer_ros. Also, you need an additional Handle...Message in SensorBridge. It is easily feasible, but I think there are no plans to mainline a new message format.

Splitting the messages is of course a work-around that requires no changes to Cartographer other than configuring num_accumulated_range_data.

[ojura]

Okay. I am looking at the pcd code in msg_conversion.cc and how PointCloud2 is defined. The support for having additional channels is there. It seems like we could easily define a new pcd point type here which would include a channel e.g. t for timing information, like here: http://pointclouds.org/documentation/tutorials/adding_custom_ptype.php

Hence this would be a minimally invasive change. If a channel t is there, we would use it, just like with the intensities channel. What do you think?

[SimoSilvere]

Hi all,

I think you could insert time offset (float type) to current type of PointCloud2 message as coordinates in point cloud library are saved as follows:

struct PointXYZ { float x; float y; float z; float padding; };

You would need to use bytes 12-15 of the message which is the padding shown above. I would like to know how to exactly use the per-point time in SensorBridge. Are the times inserted into carto::sensor::TimedPointCloud offsets? Should the time of the last point of every point cloud be 0.0 and every time offset before that negative?

[ojura]

@SimoSilvere that's exactly what I'm looking into right now. I propose we change SensorBridge::HandlePointCloud2Message, which currently inserts a relative time of zero for all points: https://github.com/googlecartographer/cartographer_ros/blob/8dc0352d43e2a91826e0aa3dbf835217529a8af8/cartographer_ros/cartographer_ros/sensor_bridge.cc#L172 to make use of an additional PointCloud2 channel called time, if it exists, similar to here where we check if the intensity channel is available, and if it is, it's used.

[ojura]

BTW, what should be stamp of the whole timed point cloud? The earliest, or the latest point? If it's the latest point, are all offsets supposed to be negative?

[ojura]

@SimoSilvere I have prepared PR #773 to go in this direction.

[ojura]

896

[andraspalffy]

Hello,

can someone point me where this per-point timestamp is used later? I see that the loading of time channel is solved with a PointXYZIT structure, but where is the transformation interpolated for each point's timestamp?

I see when it is done per each "batch", and also see the suggestion to divide the scan into as many "ROS messages" with individual timestamps as possible. E.g. here is the config param which sets how many range_data (batch) to accumulate to make a scan:

https://github.com/cartographer-project/cartographer_ros/blob/master/cartographer_ros/configuration_files/backpack_3d.lua#L46

What is the general suggestion, should per point transformation be performed (is it even feasible, and if yes, where is it implemented), or should we focus on splitting a scan into as many subscans/"ROS messages" as possible and trust the per-subscan compensation?

BTW I am using a small scanning LiDAR, similar to the Livox ones, not a rotating one.

Thank you for your advice!