How can I adapt this work to custom LiDAR dataset?

[LoyalLumber]

First of all, many thanks for this great work.

Currently, I'm trying to adapt this work to my custom LiDAR dataset including Image, pointcloud (from VLP-16), and IMU.

My goal is to build 'Semantic map based on LiDAR pointcloud' where the semantic map is online-updated and used for robot path planning. However, it is hard to figure out where I should start, and I hope to get some advices from the experts.

ANSWER: How can I change the source codes for the adaptation? (e.g., configuration of LiDAR (VLP-16), camera parameters, calibration between camera and LiDAR, etc.)

It is very fruitful if you give me some instruction (or add some guidelines to ReadME of this repository.). I believe most users have 16 or 32-channel LiDAR, not the 64-channel one (KITTI dataset provides) due to its expensive cost. Thus, such adaptation guideline will be crucial for most users.

I look forward to hearing from you. Thanks.

[YuePanEdward]

Thanks for your interest on our work. For spinning LiDARs with different channel number, you may try to change a bit in the mapper config file from:

lidar: # ouster-64 (change according to the intrinsic.yaml)
  verbosity: 1
  width: 1024  # Px
  height: 64
  fov_up: 16.6  # deg
  fov_down: -16.6
  max_range: &max_range 40.0  # m
  min_range: &min_range 1.0
  smooth_thre_ratio: 0.1 # normal estimation threshold

to something like this:

lidar: # VLP-16
  verbosity: 1
  width: 512  # Px
  height: 16
  fov_up: 15.0  # deg
  fov_down: -15.0
  max_range: &max_range 40.0  # m
  min_range: &min_range 1.0
  smooth_thre_ratio: 0.1 # normal estimation threshold

The extrinsic parameters for LiDAR can be changed from here.

However, currently, the semantic / instance label (either the ground truth or the network prediction) should be provided by the user beforehand. You may try this and this for the panoptic segmentation task and store the segmentation labels to the "label" property of the point cloud message in the ROS bag.

We will add more instructions to the Readme later.

[LoyalLumber]

Dear author, Thanks for the considerate reply.

Now, my big concern is the possible degradation of the pre-trained model in this and this when using my VLP-16 dataset. As the models from the repos are trained using SemanticKITTI (with 64channel LiDAR), the generalization for sparser LiDARs might not be smooth.

Currently, I'm not affordable to get a ground-truth label on segmentation for my VLP-16 dataset.

Can you advise me to handle this situation? Sorry for the complicated question.

I look forward to hearing from you. Thanks.

[YuePanEdward]

We haven't done experiments on VLP16 so far. It seems that there's a gap of research on the semantic segmentation for 16 beam LiDAR. You may try to use segmentation neural networks based on raw point clouds instead of range images. Domain transfer from 64 beam to lower beam LiDAR (like this) may also work.