Textured mesh artifact

[gaoalexander]

Describe the bug When I extract a textured, uv-unwrapped mesh, there appears to be an artifact which looks like a discontinuity in the mesh along the x and y axis. The mesh looks like it is separated into 4 qudrants.

To Reproduce Steps to reproduce the behavior:

1. Optimize SDF model (BakedSDF in this case)
2. Export mesh.
3. Extract textured, uv-unwrapped mesh.

Expected behavior Results are nearly as expected except for visible artifact.

Screenshots

Additional context Using default value target_num_faces=50000.

Raw mesh export (prior to re-loading to extract textured mesh) looks fine:

[niujinshuchong]

Hi, we use sliding window to extract the mesh so the mesh is not connected to each other at the window boundary. The textured mesh extraction will simplify the mesh using some algorithm so the mesh vertices position will be displaced and that's why you see it is separated into 4 parts.

But I don't why there are also many holes in the floor. Could you share the full commands to reproduce your results?

[f-dy]

@niujinshuchong can you please point to the code that does the mesh simplification? Why not stitch the sub-meshes before running the simplification?

[f-dy]

OK I think I found it, setting target_num_faces to None will skip decimation. For the second question, it seems like un-splitting vertices after mesh extraction would work.

[niujinshuchong]

@f-dy Un-splitting vertices sounds like a great idea! Could you kindly provide some reference of it? PR are also welcome.

[f-dy]

@niujinshuchong are vertices on both sides of the boundary supposed to have exactly the same coordinates? If yes, then using python's default dictionary hashing or anything that counts the number of identical elements in a list.vector is the easiest way to go.

Else, we would have to use a kdtree, put all vertices for all meshes in the kdtree (or just the boundary vertices, if we can easily identify them), then for each vertex do a ball search with a small radius (eg 1/1000 of the average edge length from the marching cubes result), and identify collocated vertices that way.

Looking at the code (here), you're running MC on volumes that don't overlap, so the boundary vertices may have wrong position and normal (to be checked). I think subvolumes should have an overlap of at least on voxels (but maybe I misread the code).

[f-dy]

To summarize, what I would recommend is:

• here use:

  x_min, x_max = xs[i]-(xs[i + 1]-xs[i])/(cropN-2), xs[i + 1]+(xs[i + 1]-xs[i])/(cropN-2)
  y_min, y_max = ys[j]-(ys[j + 1]-ys[j])/(cropN-2), ys[j + 1]+(ys[j + 1]-ys[j])/(cropN-2)
  z_min, z_max = zs[k]-(zs[k + 1]-zs[k])/(cropN-2), zs[k + 1]+(zs[k + 1]-zs[k])/(cropN-2)

• here remove all vertices that have one coordinate equal to x_min, x_max, y_min, y_max, z_min or z_max, and the corresponding faces and normals
• here write a better "concatenate" that merges collocated vertices using one of the algorithms above. I think that since we have a 2-voxel overlap between volumes, we are guaranteed to have collocated vertices (and normals), but I'm not 100% sure

[niujinshuchong]

@f-dy Thanks for the suggestion. The boundary vertices should have exactly same positions and these vertices can be merged with trimesh's merge_vertices. It's fixed in the latest commit.

However, we should run the sdf network with highest precision (use --torch-precision highest in ns-extract-mesh) to make sure the boundary vertices have the same positions, otherwise there might be some randomness.

[jaysonnguyen]

Hi, we use sliding window to extract the mesh so the mesh is not connected to each other at the window boundary. The textured mesh extraction will simplify the mesh using some algorithm so the mesh vertices position will be displaced and that's why you see it is separated into 4 parts.

But I don't why there are also many holes in the floor. Could you share the full commands to reproduce your results?

Hi @niujinshuchong, Do you have any solution to connect 4 parts of textured mesh?