search

filaPro / oneformer3d

[CVPR2024] OneFormer3D: One Transformer for Unified Point Cloud Segmentation
Other
351 stars 32 forks source link

2 class instance segmentation does not work well #84

Open accoumar12 opened 1 month ago

accoumar12 commented 1 month ago

Hello,

Our goal is to segment CAD pieces, as the one depicted below. 3D semantic segmentation worked really well with PTv3, and allowed us to separate the faces (red) from the deburring (blue). However, I can't make it to apply your framework to properly segment these pieces. I would be able to identify the recto and the verso faces as two separate instances. Here is the config that i am using : oneformer3d_1xb2_sheet_metal_sample.txt

We took as a starting point the S3dis config. We are using the semantic label 0 for the faces and 1 for the deburring. For the instance labels we use 1 for the recto, 2 for the verso and 3 for the deburring. Our model seems to not be able to semantically segment any of the deburring face (blue). Do you have any suggestion to help us ?

Notably, how should instances be numbered properly ? I have trouble understanding the real handling of instances, since there can be any number of different instances to me. As I have understood, instances ids are disjoint between semantic classes. How can we trace back the semantic label of an instance since there can be multiple instances per semantic label ?

Thanks a lot for any help !

seg_0

accoumar12 commented 1 month ago

I actually made it to get decent visualisations because i forgot to run the fix of the spconv... I can now retrieve nice instances of my recto, here we see the number of points of each instance( knowing that the initial point cloud is around 30000 points). image However, I do not understand why the model can not provide me with the verso instances,neither the deburring ones. I still would need a way to reconstruct back the whole piece with instances and semantic labels.