search

Gorilla-Lab-SCUT / SSTNet

Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks
MIT License
97 stars 8 forks source link

Would you show how SSTNet work on the S3DIS? #3

Open Taochen12138 opened 2 years ago

Taochen12138 commented 2 years ago

I only see the scannetv2 config while your article show the results on both s3dis and scannetv2. Can you release the code of s3dis?

lzhnb commented 2 years ago

Hi, thank you very much for your attention to our work. I also foresee that someone will ask S3DIS related questions, so I hope to give a unified reply to this question here.

The code of S3DIS is too messy and I do not have enough time to organize it.

I will tell you some details about the S3DIS.

  1. The dataset and preprocess API can refer to gorilla3d's preprocess and gorilla3d's datasets. The voxelization parameters follow ScanNet's, you can see it in the default parameters in S3DISInstDev
  2. The superpoints of S3DIS are generated by the superpoint-graph, you just need to download the pretrained model and follow their S3DIS.md, and you can get the superpoints of S3DIS.
  3. In the training network setting, the S3DIS's setting is strange, it treat the cell/floor/wall as instances. And we found that most scenes just have one cell and one floor, so we will group all cell/floor points as one instance, while this operation will damage the performance slightly. And for the walls, we treat them as normal object instances.(We consider that each scene may have several walls)

I feel sorry for this. And I hope these can help you.

Taochen12138 commented 2 years ago

I do appreciate for your quick reply. Now, I have already finished the second step above. The superpoints are generated. Since the scannetv2 is too large, I still downloading it and I try to apply s3dis directly to the network. Here is a special question. From 146 line to 150 line in scannetv2.py, the superpoints and xyz uses the same valid_idx array. So I assume that superpoints and xyz has the same shape. But the superpoints I generate seems have a different shape. If it means I got some mistake or misunderstanding?

Here is the code:

xyz_middle = xyz_middle[valid_idxs] xyz = xyz[valid_idxs] rgb = rgb[valid_idxs] semantic_label = semantic_label[valid_idxs] superpoint = np.unique(superpoint[valid_idxs], return_inverse=True)[1]

lzhnb commented 2 years ago

You are right. The 0-dim of superpoint ids should be equal to xyz's.

And superpoint generation in superpoint_graph will vocalize the point cloud S3DIS first, so it will downsample the number.

I guess that the superpoint_graph processing may return a point-to-voxel map, which can help you realize this.(assign the superpoint id for each point). You can read the code in superpoint_graph partition.

I'm sorry that I forget some details about it.

Taochen12138 commented 2 years ago

Thank you. I think I get what I need, like the image shows. image

yangxin6 commented 2 years ago

Thank you. I think I get what I need, like the image shows. image I’m also dealing with a dataset that is similar to S3DIS recently. Can you share your code on S3DIS datasets?

SijanNeupane49 commented 2 years ago

Hi, Could you please share your code on S3DIS datasets? I would be grateful if you could do that for me I am in dire need of the code for S3DIS dataset. Please. Thank you in advance.

lzhnb commented 2 years ago

Hi, Could you please share your code on S3DIS datasets? I would be grateful if you could do that for me I am in dire need of the code for S3DIS dataset. Please. Thank you in advance.

The code for S3DIS dataset has been released in gorilla3d.

SijanNeupane49 commented 2 years ago

@lzhnb Thank you for your quick reply. However,I have got some error and raised another issue . It would be really helpful if you could help me with that issue.

SijanNeupane49 commented 2 years ago

I do appreciate for your quick reply. Now, I have already finished the second step above. The superpoints are generated. Since the scannetv2 is too large, I still downloading it and I try to apply s3dis directly to the network. Here is a special question. From 146 line to 150 line in scannetv2.py, the superpoints and xyz uses the same valid_idx array. So I assume that superpoints and xyz has the same shape. But the superpoints I generate seems have a different shape. If it means I got some mistake or misunderstanding?

Here is the code:

xyz_middle = xyz_middle[valid_idxs] xyz = xyz[valid_idxs] rgb = rgb[valid_idxs] semantic_label = semantic_label[valid_idxs] superpoint = np.unique(superpoint[valid_idxs], return_inverse=True)[1]

@Taochen12138 How could you solve the second step? How did you generate superpoints? Please help me. Thank you in advance.

SijanNeupane49 commented 2 years ago

@lzhnb

I do appreciate for your quick reply. Now, I have already finished the second step above. The superpoints are generated. Since the scannetv2 is too large, I still downloading it and I try to apply s3dis directly to the network. Here is a special question. From 146 line to 150 line in scannetv2.py, the superpoints and xyz uses the same valid_idx array. So I assume that superpoints and xyz has the same shape. But the superpoints I generate seems have a different shape. If it means I got some mistake or misunderstanding?

Here is the code:

xyz_middle = xyz_middle[valid_idxs] xyz = xyz[valid_idxs] rgb = rgb[valid_idxs] semantic_label = semantic_label[valid_idxs] superpoint = np.unique(superpoint[valid_idxs], return_inverse=True)[1]

Thank you. I think I get what I need, like the image shows. image

Hi, how did you solve the issue as I am also getting dimension mismatch error, where it says my dimension of superpoint graph is not the same as the dimension of semantic label? It would be immense help if you could give me some suggestions. Thank you in advance.

xuxiaoxxxx commented 1 month ago

@lzhnb

I do appreciate for your quick reply. Now, I have already finished the second step above. The superpoints are generated. Since the scannetv2 is too large, I still downloading it and I try to apply s3dis directly to the network. Here is a special question. From 146 line to 150 line in scannetv2.py, the superpoints and xyz uses the same valid_idx array. So I assume that superpoints and xyz has the same shape. But the superpoints I generate seems have a different shape. If it means I got some mistake or misunderstanding? Here is the code: xyz_middle = xyz_middle[valid_idxs] xyz = xyz[valid_idxs] rgb = rgb[valid_idxs] semantic_label = semantic_label[valid_idxs] superpoint = np.unique(superpoint[valid_idxs], return_inverse=True)[1]

Thank you. I think I get what I need, like the image shows. image

Hi, how did you solve the issue as I am also getting dimension mismatch error, where it says my dimension of superpoint graph is not the same as the dimension of semantic label? It would be immense help if you could give me some suggestions. Thank you in advance.

Hi, do you slove this problem? I also meet this problem.