search

ldkong1205 / Robo3D

[ICCV 2023] Robo3D: Towards Robust and Reliable 3D Perception against Corruptions
https://ldkong.com/Robo3D
293 stars 24 forks source link

The construction issue of the KITTI-C dataset #7

Open Hongbin98 opened 6 months ago

Hongbin98 commented 6 months ago

Thanks for your work.

When I tried to explore the KITTI-C dataset, I found that the model performance of the 'wet_ground' corruption is same at all three levels. So I wonder if there are some problems in the point cloud data under this condition?

Note: I download the KITTI-C dataset from https://opendatalab.com/OpenDataLab/KITTI-C/tree/main/raw

ldkong1205 commented 6 months ago

Hi @Hongbin98, thanks for your question!

We observed similar patterns with some models in our benchmark (see our supplementary file). We conjecture that this is because the wet_ground (mainly causes missing points on the ground) is not a very sensitive type of corruption to existing 3D perception models.

Indeed, the LiDAR scenes are imbalanced towards certain majority classes, including the ground. A certain loss of LiDAR points for these classes will not likely be an issue since there are a sufficient number of points remaining during the training.