Question about the reconstruciton on ScanNet

[JIANG-CX]

Great work! I have two questions about the reconstruction on the ScanNet dataset.

1. I'm curious about the process of preparing ScanNet data. I assume that the provided scannet.ply file is generated by back-projecting the depth image using the pose information. Can you explain how you downsampled the point cloud and combined the normal and point cloud data to store them in a single .ply file?

2. I cannot reproduce the results shown in NKSR-USAGE.md. This is the code and result, should I tune some parameters?

   if __name__ == '__main__':
   warning_on_low_memory(20000.0)
   device = torch.device("cuda:0")
   
   scannet_geom = load_scannet_example()
   
   input_xyz = torch.from_numpy(np.asarray(scannet_geom.points)).float().to(device)
   input_normal = torch.from_numpy(np.asarray(scannet_geom.normals)).float().to(device)
   
   reconstructor = nksr.Reconstructor(device)
   field = reconstructor.reconstruct(input_xyz, input_normal, detail_level=1.0, chunk_size=50.0, voxel_size=0.02)
   mesh = field.extract_dual_mesh(mise_iter=2)
   
   vis.show_3d([vis.mesh(mesh.v, mesh.f)], [scannet_geom])
   vis.to_file(vis.mesh(mesh.v, mesh.f), "scannet.ply")

   

Thanks a lot.

[heiwang1997]

Hi,

For 1, the point cloud is prepared by @fwilliams, could you give some guidance here?

For 2, we disregard voxel_size and detail_level if chunk_size is provided. Could you try examples/recons_scannet.py that I've just pushed? If you really want to use chunk mode, please scale your point cloud by 5 and apply the reconstruction. This is because the voxel size during training is 0.1, and we want to simulate a voxel size 0.02 here.

[JIANG-CX]

Thank you for your answer. However, I am still confused as to why the voxel_size and detail_level are disregarded when a chunk size is provided. From my perspective, the chunk size is only used to separate the scene into multiple pieces, and there seems to be no conflict with the voxel_size and detail_level."

[heiwang1997]

Yes, you are absolutely correct. What I was trying to say is that this is not supported for now (in our TODO list actually). I'm gonna fix that soon 😊

In the meantime, if you want to fix this, you should scale your point cloud for now.

[JIANG-CX]

OK, I am looking forward to the update 😊

[H-tr]

Hi, may I know how exactly to scale the pointcloud by hand? Which function should I use?