NVlabs / nvdiffrast

Nvdiffrast - Modular Primitives for High-Performance Differentiable Rendering
Other
1.37k stars 146 forks source link

There is a problem with the color rendered in the middle of the head in the picture #152

Closed xuduo18311199384 closed 9 months ago

xuduo18311199384 commented 9 months ago

aaaaaa16 When i render a 3D head to 2D image, there is a problem with the color rendered in the middle of the head in the picture

s-laine commented 9 months ago

This is most likely caused by incorrectly merging vertices with identical positions but different texture coordinates. This causes the texture coordinate to change from, say, u=0.99 to u=0.00 at the wrap-around point, instead of advancing from 0.99 to 1.00 which would give the correct result.

You can merge the vertex positions while keeping texture coordinates separate by using different index buffers for positions and attributes.

xuduo18311199384 commented 9 months ago

Thanks for you reply! In the 'interpolate(attr, rast, tri, rast_db=None, diff_attrs=None)' function, the attr input corresponds to ‘vt-coord’ for each vertex, which does not support keeping texture coordinates separate. Although a vertex can correspond to two vts in the mesh obj, I do not know how to change it specifically during rendering steps. Can you tell me specifically how to operate it?

xuduo18311199384 commented 9 months ago

Copy another vertex so that both vertices have the same position but different texture coordinates. Then differentiate the vertex information corresponding to the triangular patch accordingly. This operation will increase the number of vertices, and due to different vertex IDs, the triangular patches will not be continuous. Is this operation a feasible solution? Is there a better solution? @s-laine

s-laine commented 9 months ago

There is no need to duplicate the vertex positions, and it's better if you don't.

Instead, you can use separate indices and data for positions and texture coordinates for each triangle. See, e.g., the render() function in the "earth" example here that takes individual pos, pos_idx and uv, uv_idx parameters.

If you are loading the mesh from an .obj file, then — roughly speaking — the v and vt arrays turn into the pos and uv arrays, and the position/texture indices from the face definitions become the index triplets in pos_idx and uv_idx arrays.

xuduo18311199384 commented 9 months ago

I am very grateful for your reply. Through this method, I have solved this problem. aaaaaa0

xuduo18311199384 commented 9 months ago

I have another question. I have an obj of a head, but it has three texture maps, two of which are for the eyeballs, and the third is for the rest of the head, as shown in the image below. How can I render the entire head at the same time? eye_ball_tex stage3_uv 2023-12-19 16-52-03 的屏幕截图

@s-laine

s-laine commented 9 months ago

There is no builtin support for rendering from multiple textures at the moment. A simple solution is to combine the textures into a single texture atlas and adjust the vertices' texture coordinates according to their new location in the atlas.

You need to be careful if your textures are wrapping, though, as you don't want to inadvertently access a neighboring texture in the atlas. In your case it looks like only the head texture is supposed to wrap and only in the x direction, so if you place the head texture and eyeballs vertically into an atlas, it should be okay. Otherwise you may need to add padding to ensure that the textures wrap correctly.

xuduo18311199384 commented 9 months ago

Yep. I already combine the two textures in the cols direction. At the same time i remap the 'vt' value to new numeric range. Next i will render based on this new UV texture. result