[Guidance needed] Running in Gazebo Simulation

[HappySamuel]

Hi @TixiaoShan

I have been trying your LIO-SAM on gazebo with the velodyne gazebo. But i got the following warning.

And the outcome of rviz become like this, exactly as per the warning descripted.

params.yaml

Is there a way to use your LIO-SAM in gazebo, is there a way not to work with lidar that doesn't output timestamp for each point?

Best, Samuel

[Pallav1299]

Considering you are collecting the data using simulation, I suppose the pointcloud is undistorted. Please verify the same. If yes, then consider it to be just a warning. Can you share some sample rosbags with IMU and LIDAR data in your simulation world?

[HappySamuel]

Hi @Pallav1299

Actually i am trying to run mapping directly with LIO-SAM in the gazebo. But i found that the pointcloud comes from the gazebo has the time for each point missing. I manage to simulate the 3d lidar from 16beam to 128beams, but unfortunately, no "time" field for the pointcloud data. Let me try to record some rosbags for you. Will send the download link here later.

Hi @TixiaoShan Is there a way to use LIO-SAM without providing the "time" field in the pointcloud data? Besides, i found a software to be used to callibrate the imu of Ouster lidar (Real hardware). Step 1: Using imu_calib to generate the calibration file of IMU. https://github.com/dpkoch/imu_calib Step 2: Using imu_complimentary_filter / imu_filter_magdwick to generate orientation (RPY) info from 6dof IMU. https://github.com/ccny-ros-pkg/imu_tools Although i have applied above items, but i yet to get the IMU extrinsics setting correctly, which eventually causing Jumpping up and down.

But for the gazebo, still hasnt settle the large velocity warning. Is there a workaround on this?

Best, Samuel

[Pallav1299]

@HappySamuel time field in the pointcloud data is used for deskewing the pointcloud. You have to check if the pointcloud data is undistorted or not. Kitti dataset also doesn't have the time field in the pointcloud data. But it is working since the pointcloud data is undistorted.

I think you should verify the same.

[jpgvl]

Hi, I have the same problem here. time field is being published but I get the same warning and the output in rviz is more or less the same than @HappySamuel image.

[jpgvl]

I'm sorry, my problem was due to the extrinsic IMU parameters. They are well explained in the README so now it goes perfect in gazebo.

[HappySamuel]

Hi @jpgvl

So you solve your issue via setting the extrinsic IMU parameters correctly inside the gazebo?

Best, Samuel

[jpgvl]

Hi @HappySamuel

No, my problem was about the extrinsic parameters in params.yaml. I changed them from the default to:

 extrinsicTrans: [-2.155, 0.0, -0.83]
  extrinsicRot: [1, 0, 0,
                 0, 1, 0,
                 0, 0, 1]
  extrinsicRPY: [1, 0, 0,
                 0, 1, 0,
                 0, 0, 1]

Where extrinsicTrans is the transformation between lidar and IMU - I got it by using rosrun tf tf_echo robot_laser_link robot_imu_link . The extrinsicRot and extrinsicRPY are explained in the README: The reason why there are two extrinsics is that my IMU (Microstrain 3DM-GX5-25) acceleration and attitude have different cooridinates. Depend on your IMU manufacturer, the two extrinsics for your IMU may or may not be the same. Maybe you will need the same as me, but check it.

[celikemir]

hi @jpgvl how did you get your extrinsicTrans matrice value and other Rot and RPY ? I have same problem.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tenghooi]

Maybe you can refer to this Duplicate of #62

[esaumar]

Hi! Has anyone have found any other reason for this behavior? Besides the extrinsic parameters in params.yaml. I am using a differential drive robot model (https://github.com/sanuann/DifferentialDriveRobot) where I attached a Velodyne VLP-16 (https://github.com/lmark1/velodyne_simulator) and a IMU (libgazebo_ros_imu.so).

I attached the Velodyne into the robot chassis using the file myrobot.xacro from https://github.com/sanuann/DifferentialDriveRobot

<xacro:include filename="$(find velodyne_description)/urdf/VLP-16.urdf.xacro"/>
  <VLP-16 parent="chassis" name="velodyne" topic="/velodyne_points" hz="10" samples="440">
    <origin xyz="0 0 0.0" rpy="0 0 0" />
  </VLP-16>

I attached the IMU into the velodyne body frame and defined an update rate of 500Hz in the file myrobot.xacro from https://github.com/sanuann/DifferentialDriveRobot

<gazebo>
    <plugin name="imu_plugin" filename="libgazebo_ros_imu.so">
      <alwaysOn>true</alwaysOn>
      <bodyName>velodyne</bodyName>
      <topicName>imu</topicName>
      <serviceName>imu_service</serviceName>
      <gaussianNoise>0.0</gaussianNoise>
      <updateRate>500.0</updateRate>
    </plugin>
  </gazebo>

Thus, all sensors are in the same body frame, velodyne. That is why I used this Extrinsics

# Extrinsics (lidar -> IMU)
  extrinsicTrans:  [0.000, 0.000, 0.0]
  extrinsicRot: [1, 0, 0,
                  0, 1, 0,
                  0, 0, 1]
  extrinsicRPY: [1,  0, 0,
                 0, 1, 0,
                 0, 0, 1]

but can't get a good reconstruction. I also tested moving the VLP-16 0.4m above the chassis and modified the Extrinsics to [0.000, 0.000, -0.438] (gotten from rosrun tf tf_echo velodyne chassis) but I have the same behavior.

Any suggestion will be appreciated. Thanks!

[Pallav1299]

@esaumar This seems to be a problem related to Z-axis acceleration from IMU. Could you share a snippet of IMU data that you are getting?

[jpgvl]

hi @jpgvl how did you get your extrinsicTrans matrice value and other Rot and RPY ? I have same problem.

Hi @celikemir sorry about the delay, I didn't notice about yor comment until now. Maybe you already know now but I'm explaining it anyway. The rotation matrix depends on the rotation of the XYZ axis of the IMU, in my case they are aligned with the XYZ of the robot, so there's no rotation and the rotation matrix is the identity. Apart from that my IMU has the same rotation for acceleration and attitude (I suppose that if it was different the manufacturer would warn you somehow, I don't think it is very common to have two different rotations). If you IMU is in gazebo you can check the driver, but if you have no idea you can move the robot and see the accelerations to know which one is in the Z axis (always more or less 9.8m/s2) and which one is bigger when moving forward (see if it is negative or possitive).

In the case of the README of this project the IMU has a picture in the corner were the X axis points backwards and the Y one points to the left (everything relative to the lidar).

You can calculate this rotation in roll pitch yaw using trigonometry and calculate the rotation matrix (I imagine that you know how but I add a screenshot from wikipedia just in case).

[esaumar]

Hi @Pallav1299 , thanks for your answer.

@esaumar This seems to be a problem related to Z-axis acceleration from IMU. Could you share a snippet of IMU data that you are getting?

Below you can find the IMU data I'm getting. I'm using libgazebo_ros_imu for the IMU, another user of LIO-SAM told me that I can try with libgazebo_ros_imu_sensor. Do you think that could help?

header: 
  seq: 14764
  stamp: 
    secs: 1947
    nsecs: 758000000
  frame_id: "chassis"
orientation: 
  x: 2.27188461314e-06
  y: -4.9678469459e-07
  z: 0.99810920732
  w: -0.0614655208814
orientation_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
angular_velocity: 
  x: 0.000769452875622
  y: 0.00324902769181
  z: 4.03222099903e-05
angular_velocity_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
linear_acceleration: 
  x: 0.227426137416
  y: 0.0242159376913
  z: 4.67897961339
linear_acceleration_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
---
header: 
  seq: 14765
  stamp: 
    secs: 1947
    nsecs: 760000000
  frame_id: "chassis"
orientation: 
  x: 3.77729827605e-06
  y: -1.12818890019e-06
  z: 0.998109202208
  w: -0.061465603808
orientation_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
angular_velocity: 
  x: -0.00145898346466
  y: -0.00644122811621
  z: 0.000125736985632
angular_velocity_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
linear_acceleration: 
  x: -0.308154769939
  y: 0.146666945668
  z: 4.17352014109
linear_acceleration_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
---
header: 
  seq: 14766
  stamp: 
    secs: 1947
    nsecs: 762000000
  frame_id: "chassis"
orientation: 
  x: 4.49033612577e-06
  y: -2.25415327875e-06
  z: 0.998109198704
  w: -0.0614656606382
orientation_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
angular_velocity: 
  x: 0.000117160434199
  y: 0.00239381430326
  z: 3.37569606052e-05
angular_velocity_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
linear_acceleration: 
  x: 0.267415818727
  y: -0.107983211178
  z: -3.9514741496
linear_acceleration_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

[esaumar]

@Pallav1299 Finally I got consistent reconstructions with LIO-SAM! You were right, it seemed that it was an issue with the IMU. The gravity should be included in the inertial measurements. For that, I changed the plugin into Gazebo, from libgazebo_ros_imu to libgazebo_ros_imu_sensor as suggested by some LIO-SAM users.

By using libgazebo_ros_imu_sensor now I can see the gravity on the Z-axis acceleration. Bad IMU

orientation: 
  x: 2.27188461314e-06
  y: -4.9678469459e-07
  z: 0.99810920732
  w: -0.0614655208814
orientation_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
angular_velocity: 
  x: 0.000769452875622
  y: 0.00324902769181
  z: 4.03222099903e-05
angular_velocity_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
linear_acceleration: 
  x: 0.227426137416
  y: 0.0242159376913
  z: 4.67897961339
linear_acceleration_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

Good IMU

orientation: 
  x: 1.55563938577e-14
  y: -6.53325214776e-15
  z: -2.5826076197e-15
  w: 1.0
orientation_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
angular_velocity: 
  x: -6.18309291924e-13
  y: 2.5967517029e-13
  z: 6.44077756609e-21
angular_velocity_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
linear_acceleration: 
  x: 1.2805174418e-13
  y: 3.04905318523e-13
  z: 9.8
linear_acceleration_covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

Thanks for your support!

[lzy-doom]

Hello. I am using libgazebo_ros_imu_sensor but the linear acceleration still does not include gravity acceleration, how to set? Thank you

[mikolodz]

@esaumar Hello, could you possibly share your gazebo project, as something that's working with LIO-SAM? It would be very helpful for others playing around to make it working too. Thank you

btw. I've spent quite a lot of time to get it working with Isaac Sim, but with no luck. I'm getting the imu preintegration errors, although using gtsam 4.0.2. I think that Isaac Sim has problems with maintaining the right publishing time. It sends couple of duplicate messages from the lidar, at the same nanosecond with different point clouds. I've lost my patience and now I want to go with Gazebo which seems to be working at least for some people out there.

[Glennine]

@mikolodz hi, have you tried it on isaac sim in the previous months? I face the same problem now.

[esaumar]

@esaumar Hello, could you possibly share your gazebo project, as something that's working with LIO-SAM? It would be very helpful for others playing around to make it working too. Thank you

btw. I've spent quite a lot of time to get it working with Isaac Sim, but with no luck. I'm getting the imu preintegration errors, although using gtsam 4.0.2. I think that Isaac Sim has problems with maintaining the right publishing time. It sends couple of duplicate messages from the lidar, at the same nanosecond with different point clouds. I've lost my patience and now I want to go with Gazebo which seems to be working at least for some people out there.

Hi @mikolodz , sorry for my late response, I missed your message. Unfortunately I had to remove my gazebo setup a couple years ago. I'm no longer testing LIO-SAM but I remember I used hector_gazebo plugins and lidar_simulator repo. I hope you can set it up successfully.

[esaumar]

Hello. I am using libgazebo_ros_imu_sensor but the linear acceleration still does not include gravity acceleration, how to set? Thank you

Hi @lzy-doom , sorry for my late response. Not sure how but you can try installing another imu plugin.