Questions about using NeRF Synthetic dataset

[ch1998]

When trying to use NeRF Synthetic dataset, such as lego, an error [Open3D Error] is reported (static std::tuple<std::shared_ptr, std::vector<double, std::allocator > > open3d::geometry::TriangleMesh::CreateFromPointCloudPoisson(const open3d::geometry::PointCloud&, size_t, float, float, bool, int)) /root/Open3D/cpp/open3d/geometry/SurfaceReconstructionPoisson.cpp:732 : Point cloud has no normals

console print Background points: torch.Size([0, 3]) torch.Size([0, 3]) torch.Size([0, 3])

The lego data have no background, but in the coarse mesh stage, bg_pcd is still processed. Is this a bug?

[xiiliu]

Meet the same problem when using my own dataset. I segment the background before I train 3d gaussian splatting model. So the background points are very few, could this be the problem?

[Anttwo]

Hello guys,

Thank you for noticing this!

Indeed, I suppose this is due to the empty background. Since we were interested in full, real scenes only, there was always a background in my scenes and I forgot to handle the case of an empty background, that could be useful for synthetic datasets. I'm sorry for that!

I just changed the coarse_mesh.py script. I did not re-test the code because this change is really minor (it was just a few 'if's to add), but it should work with empty backgrounds now. Feel free to tell me if you meet another error related to this.

Thank you!

[zhuhu00]

Does NeRF Synthesis dataset also need to generate this file with colmap?

[xiiliu]

Hello guys,

Thank you for noticing this!

Indeed, I suppose this is due to the empty background. Since we were interested in full, real scenes only, there was always a background in my scenes and I forgot to handle the case of an empty background, that could be useful for synthetic datasets. I'm sorry for that!

I just changed the coarse_mesh.py script. I did not re-test the code because this change is really minor (it was just a few 'if's to add), but it should work with empty backgrounds now. Feel free to tell me if you meet another error related to this.

Thank you!

@Anttwo Thanks a lot! It works!

[Anttwo]

Does NeRF Synthesis dataset also need to generate this file with colmap?

Oh right that is a very good point. In the original 3DGS code, the authors use a random point cloud for the synthetic scenes, and the sparse SfM point cloud for real scenes. Because I focused only on real scenes (as the problem is much more challenging), I always try to load the SfM point cloud in my code. That's why you have an error.

I should add an option to instantiate a random point cloud if no point cloud file is detected. Thank you for noticing that.

In the meantime, you can try to create a random point cloud (located inside the bounding box of the camera poses) and save it as a PLY file to replace the COLMAP point cloud, if you want (or use the COLMAP script to generate an SfM point cloud)

[ch1998]

@Anttwo Thanks a lot!