How to generate superpoints for S3DIS

[RayYoh]

Dear authors,

Thanks for your great work. As I mentioned in the title, how do you generate superpoints for S3DIS? Is there any code can be referred for the training of S3DIS?

Best

[MansiP441]

I did not use S3DIS but another custom point cloud dataset that did not have mesh. I obtained it with the following:

pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(xyz)
pcd.colors = o3d.utility.Vector3dVector(colors / 255.0)
pcd.estimate_normals()
distances = pcd.compute_nearest_neighbor_distance()
avg_dist = np.mean(distances)
radius = 1.5 * avg_dist   
mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd, o3d.utility.DoubleVector([radius, radius * 2]))
vertices = torch.from_numpy(np.array(mesh.vertices).astype(np.float32))
faces = torch.from_numpy(np.array(mesh.triangles).astype(np.int64))
superpoint = segmentator.segment_mesh(vertices, faces, 0.01, 50).numpy()

[Serissa]

I did not use S3DIS but another custom point cloud dataset that did not have mesh. I obtained it with the following:

pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(xyz)
pcd.colors = o3d.utility.Vector3dVector(colors / 255.0)
pcd.estimate_normals()
distances = pcd.compute_nearest_neighbor_distance()
avg_dist = np.mean(distances)
radius = 1.5 * avg_dist   
mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd, o3d.utility.DoubleVector([radius, radius * 2]))
vertices = torch.from_numpy(np.array(mesh.vertices).astype(np.float32))
faces = torch.from_numpy(np.array(mesh.triangles).astype(np.int64))
superpoint = segmentator.segment_mesh(vertices, faces, 0.01, 50).numpy()

Have you used the scannet_frames_25k dataset?

[MansiP441]

I did not use S3DIS but another custom point cloud dataset that did not have mesh. I obtained it with the following:

pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(xyz)
pcd.colors = o3d.utility.Vector3dVector(colors / 255.0)
pcd.estimate_normals()
distances = pcd.compute_nearest_neighbor_distance()
avg_dist = np.mean(distances)
radius = 1.5 * avg_dist   
mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd, o3d.utility.DoubleVector([radius, radius * 2]))
vertices = torch.from_numpy(np.array(mesh.vertices).astype(np.float32))
faces = torch.from_numpy(np.array(mesh.triangles).astype(np.int64))
superpoint = segmentator.segment_mesh(vertices, faces, 0.01, 50).numpy()

Have you used the scannet_frames_25k dataset?

No. It was not the scannet_frames_25k dataset.

[Serissa]

I did not use S3DIS but another custom point cloud dataset that did not have mesh. I obtained it with the following:

pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(xyz)
pcd.colors = o3d.utility.Vector3dVector(colors / 255.0)
pcd.estimate_normals()
distances = pcd.compute_nearest_neighbor_distance()
avg_dist = np.mean(distances)
radius = 1.5 * avg_dist   
mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd, o3d.utility.DoubleVector([radius, radius * 2]))
vertices = torch.from_numpy(np.array(mesh.vertices).astype(np.float32))
faces = torch.from_numpy(np.array(mesh.triangles).astype(np.int64))
superpoint = segmentator.segment_mesh(vertices, faces, 0.01, 50).numpy()

Have you used the scannet_frames_25k dataset?

No. It was not the scannet_frames_25k dataset.

What files do you use from the custom point cloud dataset? You have the RBG file? Depth file? Instance segmentation file?

[MansiP441]

I had each point cloud defined in a .npy file. Each point in the point cloud was represented by xyz and rgb values.

[Serissa]

torch.save((coords, colors, superpoint, sem_labels, instance_labels), fn[:-15] + '_inst_nostuff.pth') What is the content of the sem_labels and the instance_labels label file ?

[Serissa]

I did not use S3DIS but another custom point cloud dataset that did not have mesh. I obtained it with the following:

pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(xyz)
pcd.colors = o3d.utility.Vector3dVector(colors / 255.0)
pcd.estimate_normals()
distances = pcd.compute_nearest_neighbor_distance()
avg_dist = np.mean(distances)
radius = 1.5 * avg_dist   
mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd, o3d.utility.DoubleVector([radius, radius * 2]))
vertices = torch.from_numpy(np.array(mesh.vertices).astype(np.float32))
faces = torch.from_numpy(np.array(mesh.triangles).astype(np.int64))
superpoint = segmentator.segment_mesh(vertices, faces, 0.01, 50).numpy()

mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd, o3d.utility.DoubleVector([radius, radius * 2])) The code execution is time-consuming. Is there any way to speed up the operation?

[wdczz]

mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd, o3d.utility.DoubleVector([radius, radius * 2]))

Hello, do you get the superpoint of s3dis? QAQ

[Serissa]

mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd, o3d.utility.DoubleVector([radius, radius * 2]))

Hello, do you get the superpoint of s3dis? QAQ

I'm slow to generate superpoint files with RGB and depth files. What is the role of the superpoint file? Is there any mention of it in the article?

[xuxiaoxxxx]

The superpoint generate is normal? Following the above code I can't seem to get the exact superpoint.

[wdczz]

I can the good superpoint from Box2Mask, you can have a try.

[xuxiaoxxxx]

I can the good superpoint from Box2Mask, you can have a try.

Thanks for your reply! I read the s3dis.readme of the Box2Mask and download the oversegmentation of s3dis. The code for loading dataset is as follows:

path_sup = 'segment_labels\learned_superpoint_graph_segmentations\Area_1.conferenceRoom_1.npy'
segments_labels_info = np.load(path_sup, allow_pickle=True).item()
segments = segments_labels_info['segments']
per_point_segment_instanceID = segments_labels_info['per_point_segment_instanceID']
seg2instanceID = segments_labels_info['seg2instanceID']
per_point_segment_semanticID = segments_labels_info['per_point_segment_semanticID']
seg2semanticID = segments_labels_info['seg2semanticID']

And I guess the ''segments '' is the superpoint corresponding to the s3dis dataset?

[xuxiaoxxxx]

I can the good superpoint from Box2Mask, you can have a try.

Thanks for your reply! I read the s3dis.readme of the Box2Mask and download the oversegmentation of s3dis. The code for loading dataset is as follows:

path_sup = 'segment_labels\learned_superpoint_graph_segmentations\Area_1.conferenceRoom_1.npy'
segments_labels_info = np.load(path_sup, allow_pickle=True).item()
segments = segments_labels_info['segments']
per_point_segment_instanceID = segments_labels_info['per_point_segment_instanceID']
seg2instanceID = segments_labels_info['seg2instanceID']
per_point_segment_semanticID = segments_labels_info['per_point_segment_semanticID']
seg2semanticID = segments_labels_info['seg2semanticID']

And I guess the ''segments '' is the superpoint corresponding to the s3dis dataset?

I also visualized ''segments '‘ as shown below. Does the visualization look normal to you?

[wdczz]

The Box2Mask superpoint (Area_2_storage_5_inst_nostuff.pth) The superpoint are generated by the segmentator. (This is the same method as ScanNetv2) The instance label

[xuxiaoxxxx]

The Box2Mask superpoint (Area_2_storage_5_inst_nostuff.pth) The superpoint are generated by the segmentator. (This is the same method as ScanNetv2) The instance label

If it is convenient for you, could you please share a copy of the processed superpoint of s3dis dataset and processing code with me? My email is 2777096001@qq.com. Thank you very much!