Observe many seams

[pean1128]

After mesh pass, I found there are many seams between paired UV patches although the UV coordinates are all located within the patch border. Can you provide some advice?

[jmunkberg]

Hello,

Please refer to Section 3.2, Texturing and Fig 6 in our paper https://nvlabs.github.io/nvdiffrec/assets/paper.pdf , where this issue is discussed.

Once the topology and MLP texture representation have converged, we re-parametrize the model: we generate unique texture coordinates using xatlas and sample the MLP on the surface mesh to initialize 2D textures, then continue the optimization with fixed topology. Referring to Figure 6, this effectively removes texture seams introduced by the (u, v)-parametrization

[jmunkberg]

So, seams are expected directly after the first pass, but most seams should be gone after the second pass.

[pean1128]

So, seams are expected directly after the first pass, but most seams should be gone after the second pass.

Yeah, I have noticed in this section, but the above snapshot comes from the triangle mesh tuned after the second pass. Actually, this issue still looks obvious in other customer data as follows

[jmunkberg]

Sorry, I answered too quickly (I thought you meant after the first pass). We haven't looked closely at this. The UVs are generated by xatlas (external tool), and we currently do not explicitly regularize the UVs in the second pass.

Perhaps adding some regularization on the UVs or the mesh (triangle area, edge length, Laplacians) may help in this use case.

Another approach may be to explicitly remesh and UV map the DMTet-generated mesh with some off-the-shelf remeshing tool, like Simplygon, and feeding that output into the second pass of optimization, as the DMTet output contains a fair bit of sliver triangles.

[pean1128]

Thanks for your reply, I got that😀

[TaroPlay]

hello，pean1128. I have a question to ask you : how to get the coffee bottle result?(is it a mesh.obj file ?) Can you tell me what should I input to the network. I am really look forward to your reply . thanks !

[Jerrypiglet]

Hi @jmunkberg , I noticed the same issue about seams, and I think this is might be related to the dr.rasterize function here (by the end of the first stage of training).

If you export the rast object into a uv map and load into Blender together with the .obj from first stage, you will notice some of the vertices fall out of the valid space of the acquired UV map (see image below; the top selected vertex on the left). Not sure if this is an issue with nvdiffrast, or with the way we feed the mesh into dr.rasterize? Any ideas will be appreciated.

[TaroPlay]

你好，朱煜俊已收到你的来件，谢谢