search

Anttwo / SuGaR

[CVPR 2024] Official PyTorch implementation of SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering
https://anttwo.github.io/sugar/
Other
2.31k stars 175 forks source link

To SuGaR-er: the points regularized by SuGaR. very goooooooood! #48

Open yuedajiong opened 10 months ago

yuedajiong commented 10 months ago

image

yuedajiong commented 10 months ago

If the algorithm has mask-logic, it should be perfect.

yuedajiong commented 10 months ago

image

XOURTNEY commented 10 months ago

The reconstruction of this lion is amazing! I noticed some white lines between some of the triangles. Do you know how to weaken these white lines?

Anttwo commented 10 months ago

Thank you so much @yuedajiong for your nice words! Indeed, your lion is very cool!

Hello @XOURTNEY, The white lines between the triangles are due to the Texture interpolation method you use as well as our UV unwrapping method, which is currently really dumb.

To compute a texture for a mesh, we first need to "unwrap" the mesh with a UV map, which maps every triangle to a set of pixels in the texture image. Generally, you try to preserve some spatial consistency between the 3D geometry and the UV map: two triangles that are close to each other in the 3D mesh should also be close to each other in the UV map. But this problem is not trivial to solve!

Consequently, here, we propose a very simple (but very dumb) way to unwrap the mesh: we just map all triangles next to each other, at random locations in the UV map. It works well, but it can produce artifacts depending on the texture interpolation method you use in your software. Texture interpolation is the method you use to render the color of a triangle on the 3D mesh, using the pixel colors in the texture: Should you perform linear interpolation between the colors of neighbor pixels in the texture ("Linear" setting inside Blender) or should you just use the color of the closest pixel in the texture ("Closest" setting inside Blender)? You can see the difference in Blender: If you use "Closest", the texture will be rendered perfectly fine. But if you use "Linear", you will see some white lines between triangles, because Blender interpolates between the color of the right pixels and the color of wrong pixels in the neighborhood (i.e. pixels close to the right pixel in the UV map, but that map to totally different triangles on the mesh).

I suppose your rendering software uses Linear Texture interpolation by default. Try to use "Closest pixel" Texture interpolation and the texture should be rendered perfectly fine!