How to process noisy lidar points cloud?

[szhang963]

Hi, I have some questions for you about lidar data.

1. It is difficult to merge multi-frame pcd (resulting in a misaligned geometry due to inaccuracy poses). How can I process these data or can I use single-frame pcd and optimize the pose to enable accurate geometry in all frames？
2. If the multi-frame pcd can be merged but misaligned with image strictly, the depth supervision is usually not accurate. Does it affect the quality of RGB and depth, and how can I optimize this question?

Thanks in advance.

[georghess]

Hi! Do I understand correctly that you wonder how neurad handles noisy pose data for lidar? During training, we handle the point clouds in the sensor frame and move them to the world frame using the lidar pose. You can optimize the sensor poses during training by setting the camera_optimizer mode to S03xR3 (see here for instance). The name camera_optimizer is for compatability with nerfstudio code, but it optimizes all sensor poses (including lidar).

[szhang963]

Hi, thanks for your reply. I mean the noise is the misalignment of single-frame lidar points with images due to time synchronization, bad lidar poses, e.g. However, I found it is not obvious for the pandaset dataset. Do you know what process the pandaset makes for the lidar data in the provided data?

Thanks in advance.