search

nerfstudio-project / nerfstudio

A collaboration friendly studio for NeRFs
https://docs.nerf.studio
Apache License 2.0
9.57k stars 1.31k forks source link

Gaps after "ns-export marching cubes" on the outer surface found using neus-facto from nerfstudio #2721

Open peta-peta opened 10 months ago

peta-peta commented 10 months ago

Describe the bug Extracted model shows lined gaps on the surface. See Picture below:

image

To Reproduce

  1. Executed the following command on my trained model with neus-facto: ns-export marching-cubes --load-config outputs/Room/config.yml --output-dir Room 2.open3d draw Room/mesh.obj

Expected behavior I would have expected a watertight closed surface. No weird artifacts, when I open sdf_marching_cubes_mesh.ply of the same trained and extracted model. Btw. I noticed that the same problem appears also to the standard replica-room0 case. No closed surface found! the rendered model has gaps.

ktertikas commented 10 months ago

Hi, I am observing a similar issue. This is probably caused by the fact that the default grid resolution in the exporting script is 1024: https://github.com/nerfstudio-project/nerfstudio/blob/05d305467f92cdcd2905de480f316b483d7ecb77/nerfstudio/scripts/exporter.py#L391

At the same time, the generate_mesh_with_multires_marching_cubes function splits the space into chunks of resolution 512x512x512: https://github.com/nerfstudio-project/nerfstudio/blob/05d305467f92cdcd2905de480f316b483d7ecb77/nerfstudio/exporter/marching_cubes.py#L181

This probably causes the gaps that you are looking at. At the cost of memory, you can instead modify the generate_mesh_with_multires_marching_cubes, and keep in memory a grid that stores all the density values, running marching cubes as a final step on the entire resolution.