Problem using other LiDAR

[carlostojal]

Hi. Maybe @RuslanAgishev or @peci1 can help me out in this one. I was testing this package, and it worked with the simulated LiDAR from Gazebo, which I think is a Ouster OS1-64. However, the width and height of the pointcloud don't match up with the datasheet, even regarding the count:

$ rostopic echo /points | head -n 30
header: 
  seq: 3
  stamp: 
    secs: 2746
    nsecs: 965000000
  frame_id: "ouster_link"
height: 1
width: 70572
fields: 
  - 
    name: "x"
    offset: 0
    datatype: 7
    count: 1
  - 
    name: "y"
    offset: 4
    datatype: 7
    count: 1
  - 
    name: "z"
    offset: 8
    datatype: 7
    count: 1
  - 
    name: "intensity"
    offset: 16
    datatype: 7
    count: 1
  -

Sometimes I get this width, other times I get 70591.

I am trying to use a rosbag which also has a OS1, this with a beam count of 2048 and 128 channels (double checked with the topic width and height), but it ain't working.

Do I need to do some "reshaping" and/or downsampling to use other LiDAR models? I found some configurations on launch files mentioning number of channels and beam count, but they don't match up with the pointcloud published by Gazebo, which is working.

Thanks in advance.

[carlostojal]

Another question: as to my understanding, the semantic segmentation is done on depth images, so this module performs the LiDAR projection to a depth image. May I run into problems if I use solid-state LiDARs and/or "oddly-shaped" pointclouds (i.e. pointclouds generated by aligning several scans and sensors)? I saw you also have geometric-based traversability estimation. Would that work on any pointcloud? How accurate is it compared to the segmentation-based?

[peci1]

Dear Carlos, regarding the pointcloud shape, it looks that an unstructured pcl is published. There are two types of pointclouds: 1) structured are those you are used to, i.e. non-unit height, and they can contain NaNs (for rays that did not return). 2) unstructured pcls have height equal to 1 a width equal to the number of valid (non-NaN) points. They do not contain NaNs as only valid measurements are represented. These two representations are convertible to each other if you have some metadata.

[RuslanAgishev]

Hi Carlos, Indeed, we do the projection of an unstructured point cloud into the range image representation using the cloud_proc package. It is done due to the choice of a CNN architecture performing range image semantic segmentation. An example is in the semantic_traversability.launch.

One can launch it by specifying an input point cloud topic: roslaunch traversability_estimation semantic_traversability.launch input:=/points

Regarding your second question, geometric traversability estimation would require parameter tuning for different kinds of LiDARs. The corresponding launch file (geometric_traversability.launch) has a similar interface. We did not experiment with solid-state lidars for traversability estimation. According to our experience, semantic segmentation-based traversability estimation could be less robust for the environment scenarios not present in a training data set. However, it handles soft terrain (like vegetation) better than the geometric-based method. The two methods could be used separately or complementary depending on your application.

[carlostojal]

Dear Carlos, regarding the pointcloud shape, it looks that an unstructured pcl is published. There are two types of pointclouds: 1) structured are those you are used to, i.e. non-unit height, and they can contain NaNs (for rays that did not return). 2) unstructured pcls have height equal to 1 a width equal to the number of valid (non-NaN) points. They do not contain NaNs as only valid measurements are represented. These two representations are convertible to each other if you have some metadata.

So, does the package expect unstructured pointclouds? And do I need to specify any parameters in launch files?

[carlostojal]

Hi Carlos, Indeed, we do the projection of an unstructured point cloud into the range image representation using the cloud_proc package. It is done due to the choice of a CNN architecture performing range image semantic segmentation. An example is in the semantic_traversability.launch.

One can launch it by specifying an input point cloud topic: roslaunch traversability_estimation semantic_traversability.launch input:=/points

Regarding your second question, geometric traversability estimation would require parameter tuning for different kinds of LiDARs. The corresponding launch file (geometric_traversability.launch) has a similar interface. We did not experiment with solid-state lidars for traversability estimation. According to our experience, semantic segmentation-based traversability estimation could be less robust for the environment scenarios not present in a training data set. However, it handles soft terrain (like vegetation) better than the geometric-based method. The two methods could be used separately or complementary depending on your application.

My question regarding the projection is because I am using solid-state LiDARs with a non-repetitive pattern (Livox Horizon). Furthermore, I am aligning the scans of different LiDARs and pointclouds de-projected from stereo cameras over time and space, which results in a pointcloud which will need to be cropped and will result in a inconsistent depth image after projection. My question about the geometric-based method was for these reasons, but we will test both and see. Our scenario is also a forest, so the Rellis dataset kind of represents well our data.

[RuslanAgishev]

The semantic segmentation traversability node expects a range image as input according to the scheme given in the picture (a network architecture we trained is the DeepLab-V3 available in torchvision.):

We map Rellis-3D classes into traversable / non-traversable representation.

[carlostojal]

The semantic segmentation traversability node expects a range image as input according to the scheme given in the picture (a network architecture we trained is the DeepLab-V3 available in torchvision.):

We map Rellis-3D classes into traversable / non-traversable representation.

I see, thanks for the clarification. By the other images I also saw you do some sort of box cropping on the pointcloud before projection, which is done by the cloud_proc package. Where are the husky bags (and the other bags) of the demo located? I checked the RELLIS-3D dataset but couldn't find it.

[RuslanAgishev]

Hi. Apologies for the long response. We added the link to the bag-file sequences: http://subtdata.felk.cvut.cz/robingas/data/ Please, refer to the updated documentation.