About livox lidar

[KkCabin]

Hello, @davidskdds Thank you for your excellent work. May I know what to try if I want to try this method on a livox series lidar?

[davidskdds]

Hello @KkCabin ,

thank you for the feedback!

I already wanted to create a configuration for livox lidars in the past. My problem was that I didn't want to include any custom ros message types (which are based on other repositories) especially for a single LiDAR manufacturer. The PointCloud2 message that was provided by the Livox driver did not contain timestamps per point.

I just did some research. There is now a new Livox ros driver https://github.com/Livox-SDK/livox_ros_driver2 that supports ros and ros2.

The new driver is able to publish PointCloud2-messages with timestamps per point, if you configure it to publish "Livox pointcloud2 (PointXYZRTLT) point cloud format" (see the README.md).

Unfortunately, I do not own a Livox LiDAR. If you can provide me a short rosbag (1-2 min, with motion) containing the Livox point clouds in pointcloud2 (PointXYZRTLT) point cloud format, I can add a configuration for Livox LiDARs to the SLAM within 1-2 weeks.

[KkCabin]

Thank you for your prompt response. Unfortunately, I only have some livox bag data and don't actually have the device. Can we keep this issue open? I'm hoping that others who are interested might provide a livox bag in the PointXYZRTLT format.

[davidskdds]

Of course, we can.

I have one thing to add:

If you have already some livox rosbags, that contain livox point clouds in PointCloud2 format without timestamps per point, you can try to process them with the setting sensor: "unknown". With this setting it is assumed that the point order in the PointCloud2 message correlates with the point stamps and some pseudo point stamps are generated. The quality will decrease a little bit as this assumption does not fit always but it should work in most cases. Also change the parameter in the slam_settings.yaml so that the system can better handle the sparsity of livox point clouds: max_num_points_per_scan: 1500.

I just tried it for the rosbag "mid40_hall_example.bag" provided here and it works:

[KkCabin]

Hello, I greatly appreciate your prompt responses every time. Following your guidance, I was able to achieve results similar to what you've mentioned. However, I'm encountering some issues when attempting to use the IMU. I suspect the problem arises because the point cloud's time is estimated as the current time, whereas the IMU's time corresponds to the time when the data was logged in the bag. Here is the data I've provided that includes the IMU.

[davidskdds]

Hello @KkCabin ,

thank you for providing the data. I just had a look on your rosbag.

When I run the SLAM with IMU data, then it complains, that there is a big time difference between IMU stamps and LiDAR stamps and therefore it ignores the IMU data.

So I had a closer look on the timestamps of the IMU and LiDAR data provided in your rosbag. Therefore I compared the timestamps provided in the headers of the PointCloud2 and IMU messages.

My setup for the comparison was as follows:

In the upper terminal I just replayed the rosbag and provide a roscore. In the lower terminals I just do "rostopic echo" to the PointCloud2 and IMU topic and print in each case the parts of the rostopics that contain the word "sec". When I play the rosbag and pause it after some seconds, I get the following result:

If you compare the "sec" values of the timestamps in the first line of the lower terminals you can see that there is a huge difference > 1e7 seconds between the topics.

I hope I can help you with this.

[KkCabin]

Hello, @davidskdds, I sincerely thank you for your detailed and patient analysis, which has helped me a lot. The package I provided was obtained a long time ago by converting from LVX format files to pointcloud format through livox_ros_driver. I carelessly neglected to check the time difference between the IMU and lidar. Just now, I reconverted it and checked the consistency of the timing. Now, your program runs very well. Thank you for your outstanding work! I also deeply apologize for my oversight, which wasted your time. Thank you again!

[davidskdds]

Thank you again for the positive feedback! Don't worry, I really appreciate the interest and I believe that this kind of feedback is always helpful to improve the software.

[schablonmassig]

Hi! First off - I sincerely thank you for open sourcing this, as well as your intent to help out livox users as well. Ill be taking it out for a spin in the coming weeks.

Id like to chime in on the livox situation by refering to how it was dealt with in (D-LIO ) and in particular, this response, where kennyjchen concludes that targeting the PointXYZRTLT format is not feasible due to bugs in timestamping. To avoid dependency issues, D-LIO use a specific branch for livox support instead.

Another way to support it would be a freestanding node for doing the conversion from custommsg to the pcl format. Id be happy to provide one if that path is interesting.

[davidskdds]

Hello @schablonmassig ,

thank you for your interest and you offer to support with the Livox-LiDAR.

If you can provide a rosbag in the PointXYZRTLT format, I would be happy to test it! I took a look at your linked issue contribution. If the time stamps per point are actually incorrect, that would be really bad. There was a commit in livox_ros_driver2 10 months ago that was supposed to add the timestamps per point. I think it makes sense to test it again anyway.

If this option doesn't work, I would prefer the solution you offered with a separate node that performs the conversion. It would be really great, if you can provide such a node. I can then also add a reference to your conversion node in the readme.

Many thanks in advance

[nowPB]

Hello @schablonmassig ,

thank you for your interest and you offer to support with the Livox-LiDAR.

If you can provide a rosbag in the PointXYZRTLT format, I would be happy to test it! I took a look at your linked issue contribution. If the time stamps per point are actually incorrect, that would be really bad. There was a commit in livox_ros_driver2 10 months ago that was supposed to add the timestamps per point. I think it makes sense to test it again anyway.

If this option doesn't work, I would prefer the solution you offered with a separate node that performs the conversion. It would be really great, if you can provide such a node. I can then also add a reference to your conversion node in the readme.

Many thanks in advance Here is a package recorded with mid360. I set the xfer parameter of livox_ros_driver2 to 0.

[davidskdds]

Hello @nowPB ,

thank you a lot for providing the Livox rosbag in the PointXYZRTLT format! I just had a look at it. So, as @schablonmassig said, I can confirm there must be a bug in the driver as the per point timestamps of the Livox PointCloud2-messages are incorrect. After analyzing the per point timestamps of a single PointCloud2-message in your provided rosbag I have noticed that the per point timestamps seem to be plausible if you multiply them with 1e-9. After multiplying them with 1e-9 the result of max stamp minus min stamp of a PointCloud2-message is always around 0.1 s. For me it looks like the timestamps are provided in nanoseconds by the livox driver, although the readme says, they are provided in seconds.

As a workaround I added a configuration to this repo that handles the PointXYZRTLT nanosecond stamps, as well as a configuration to handle PointXYZRTLT in seconds, for the case the bug will be fixed. I added some more information about that in the README of this repo.

So I was able now to process your sample data and got plausible results (LiDAR only):

BUT when I added the IMU data in the configuration the results got worse. To check the reason for that I took a look at the raw IMU measurements provided in the rosbag by rostopic echo. I got the following output within the first second of the recording, when the LiDAR was static:

The problem here is that the linear acceleration does not measure the gravity as expected. In a static scenario the norm of the linear acceleration should be around 9.8 m/s^2. In the IMU data provided in the rosbag the norm is around 1. So maybe there is a unit missmatch and the acceleration is provided in g's, but actually the unit should be m/s^2.

Best regards David

[nowPB]

Hello @nowPB ,

thank you a lot for providing the Livox rosbag in the PointXYZRTLT format! I just had a look at it. So, as @schablonmassig said, I can confirm there must be a bug in the driver as the per point timestamps of the Livox PointCloud2-messages are incorrect. After analyzing the per point timestamps of a single PointCloud2-message in your provided rosbag I have noticed that the per point timestamps seem to be plausible if you multiply them with 1e-9. After multiplying them with 1e-9 the result of max stamp minus min stamp of a PointCloud2-message is always around 0.1 s. For me it looks like the timestamps are provided in nanoseconds by the livox driver, although the readme says, they are provided in seconds.

As a workaround I added a configuration to this repo that handles the PointXYZRTLT nanosecond stamps, as well as a configuration to handle PointXYZRTLT in seconds, for the case the bug will be fixed. I added some more information about that in the README of this repo.

So I was able now to process your sample data and got plausible results (LiDAR only):

BUT when I added the IMU data in the configuration the results got worse. To check the reason for that I took a look at the raw IMU measurements provided in the rosbag by rostopic echo. I got the following output within the first second of the recording, when the LiDAR was static:

The problem here is that the linear acceleration does not measure the gravity as expected. In a static scenario the norm of the linear acceleration should be around 9.8 m/s^2. In the IMU data provided in the rosbag the norm is around 1. So maybe there is a unit missmatch and the acceleration is provided in g's, but actually the unit should be m/s^2.

Best regards David

thank you !

[schablonmassig]

Hi again, and sorry for not getting back sooner. Good job nowPB, and for the thorough analysis by David.

Regarding the IMU units, the only official confirmation I've found is the added issue, where Livox confirms that the unit is in g's. (https://github.com/Livox-SDK/livox_ros_driver/issues/63).

Heres the commit where the unit problem is handled in DLIO. https://github.com/vectr-ucla/direct_lidar_inertial_odometry/commit/310a723821c325082426f2cedf3d89d7244b2ca7

[davidskdds]

Hello, thank you for the information about the Livox IMUs, so it is not a bug but a feature :-) I added a parameter in the config file to set IMUs acceleration unit to g and tested the setup successfully with the provided rosbag. I think the processing of Livox data should now work without any problems.

[schablonmassig]

I did some testing and can confirm that it works great, with the IMU, using xfer = 0. Thank you David!