NVlabs / nvdiffrec

Official code for the CVPR 2022 (oral) paper "Extracting Triangular 3D Models, Materials, and Lighting From Images".
Other
2.09k stars 222 forks source link

self-intersection in generated mesh #102

Open SarahWeiii opened 1 year ago

SarahWeiii commented 1 year ago

Hi, thank you for the excellent work! But I found self-intersections in the generated mesh, and I wonder why the mesh generated by marching tet is not manifold.

jmunkberg commented 1 year ago

Thanks @SarahWeiii ,

Note that deep marching tets is only applied in the first optimization pass (result is stored in the dmtet_mesh folder). In the second optimization pass, we lock topology and continue optimizing the vertex positions. In this second pass, there are possibilities for self-intersections. To reduce this, you can increase the amount of Laplacian regularization or use the flag lock_pos : true to disable optimization of the geometry positions in the second pass.

Relevant flags to play with:

    FLAGS.lock_pos            = False                    # Disable vertex position optimization in the second pass
    FLAGS.laplace             = "relative"               # Mesh Laplacian ["absolute", "relative"]
    FLAGS.laplace_scale       = 10000.0              # Weight for Laplace regularizer. Default is relative with large weight
zoezhou1999 commented 1 year ago

Hi @jmunkberg I am a bit new. so for topology, you mean deforming mesh using neural SDF will not lead to self-intersection, but directly optimizing vertices may produce self-intersection? Thank you!

jmunkberg commented 1 year ago

Yes, in the second pass, we move the vertex positions of a mesh with fixed topology, so if we move the vertices too much, self-intersections may occur. That is why Laplacian regularization is essential in the second pass. However, it may be tricky to dial in the amount of Laplacian regularization. Too much, and the mesh vertices wont move at all. Too little regularization, and there may be large vertex movements, producing self-intersections.