search

PRBonn / semantic-kitti-api

SemanticKITTI API for visualizing dataset, processing data, and evaluating results.
http://semantic-kitti.org
MIT License
755 stars 186 forks source link

Data alignment with pointnet/pointnet++ #106

Closed chengengliu closed 1 year ago

chengengliu commented 1 year ago

Hi, thanks a lot for your work. I'm trying to use pointnet on SemanticKitti and I am wondering how you guys preprocess the SemanticKitti data so that it fits PointNet/PointNet++? Since each point in default PointNet architecture is represented as 9-dim vector and in your data it is 4-dim. How did you convert the 4-dim to 9-dim? Any tips would be appreciated, thanks.

shayannikoohemat commented 1 year ago

PointNet uses X,Y,Z, X',Y',Z',R,G,B as the 9 dimensions. X', Y', Z' are the normalized X, Y, Z respect to the center of the room, you can do the same with kitti dataset to add three dimensions to xyz. However, you are missing colors, either you can use the intensity and repeat it as three channels to have 9 dimensions or you should change the data loader in PointNet to make it work with 4 channels (more work to be done). Alternatively, you can look at other segmentation models which use PointNet as the backbone and are tested on semantic kitti. Maybe VoxelNet, DGCNN can help.

chengengliu commented 1 year ago

PointNet uses X,Y,Z, X',Y',Z',R,G,B as the 9 dimensions. X', Y', Z' are the normalized X, Y, Z respect to the center of the room, you can do the same with kitti dataset to add three dimensions to xyz. However, you are missing colors, either you can use the intensity and repeat it as three channels to have 9 dimensions or you should change the data loader in PointNet to make it work with 4 channels (more work to be done). Alternatively, you can look at other segmentation models which use PointNet as the backbone and are tested on semantic kitti. Maybe VoxelNet, DGCNN can help.

Hi, thanks so much for your suggestion! I am wondering for your first suggestion, if repeating the intensity to fill the rest dimensions, will PointNet be effective in that case?

shayannikoohemat commented 1 year ago

I suggested repeat intensity only to fill two extra channels so (X, Y, Z, X', Y', Z', I, I, I) not all 6 remainings. The intensity is helpful as some traffic signs are reflective and the model can use that to learn. This is the fastest way you can try PointNet on sem. kitti but obviously not as efficient to modify the code to work with 4 channels.

chengengliu commented 1 year ago

I suggested repeat intensity only to fill two extra channels so (X, Y, Z, X', Y', Z', I, I, I) not all 6 remainings. The intensity is helpful as some traffic signs are reflective and the model can use that to learn. This is the fastest way you can try PointNet on sem. kitti but obviously not as efficient to modify the code to work with 4 channels.

Thank you so much for your help!