Wrong rasterization on some faces

[dragonwarrior]

when i rasterized a model into image and visualize the visible faces, i found that some front face missed which should be rasterized into pixels. I tried dr.RasterizeGLContext and dr.RasterizeCudaContext, but no difference. Following are the codes, and two views of the same model which only consist faces that are visible in one image rasterization.

    def transform_pos(self, mvp, vertices):
        """ transform vertex positions to clip space
        """
        pos = vertices
        assert pos.shape[1] == 3, "invalid shape of vertices"
        # (x,y,z) -> (x,y,z,1)
        pos = torch.cat([pos, torch.ones([pos.shape[0], 1]).to(self.device)], dim=1)
        return torch.matmul(pos, mvp.t())[None, ...]

    def render(self, mvp, vertices, faces, vertex_colors, resolution=(2048, 2048)):
        pos_clip    = self.transform_pos(mvp, vertices)
        rast_out, _ = dr.rasterize(
            self.glctx, pos_clip, faces, resolution=resolution)

        visible_tri_ids = torch.unique(rast_out[:, :, :, 3])

        mesh = trimesh.Trimesh(vertices=vertices, faces=faces[visible_tri_ids], vertex_colors=vertex_colors)
        mesh.export('visible-face-model.ply')

[s-laine]

It looks like the faces tensor you use when rendering doesn't contain all the triangles in the mesh. Perhaps it was truncated, or something else went wrong when originally loading the mesh? If the original mesh is composed of arbitrary polygons, this could be a problem with polygon triangulation in your mesh loader, as it looks like many quads have one triangle missing.

[dragonwarrior]

1. This mesh model only contains triangles
2. The number of triangles in faces is the same with that when loaded using MeshLab or Trimesh which both render correctly from the same view direction.

[s-laine]

If you have tried both dr.RasterizeGLContext and dr.RasterizeCudaContext, and these two completely unrelated rasterizers produce the same output, the problem must be in the data you're supplying to them. Perhaps you could experiment with a much simpler mesh first (say, a cube) so that you can manually examine the rasterizer input tensors and see what goes wrong.

[dragonwarrior]

hi, @s-laine Thank you for your help! I have checked the faces, but not found any special. what's more, when i sorted faces by z-coordinate of centers, the rasterized result is better than before for fewer missed faces.

[s-laine]

Could you attach position and triangle tensors, as supplied to dr.rasterize(), of a preferably simple mesh that doesn't rasterize as you'd expect?

[dragonwarrior]

ok, i prepared test code and data that you can run it. you can uncomment some lines to visualize 3d mesh and interact with it by dragging mouse. rast-debug.zip

[s-laine]

The zip (172 bytes) contains nothing but an empty folder.

For debugging I'll need the pos and tri tensors supplied to dr.rasterize() from a view that exhibits missing triangles. You can export them using pickle.dump() or torch.save(). The program code is not necessary or useful here.

[dragonwarrior]

i am very sorry, my fault. i updated previouse zip file, please check again.

[s-laine]

The triangle array contains 2000 triangles that refer to 1062 unique vertices out of the 9852 vertices in the pos array. Either there are lots of unused vertices in pos, or the triangle array is somehow incomplete.

According to your code, rasterizing these 2000 triangles produces 1903 unique triangle IDs in the output image. This sounds completely normal — some of the triangles will be hidden because they are behind other triangles, and some may be small enough after projection that they don't rasterize into any pixels.

In other words, everything seems to work as expected.

[dragonwarrior]

we also think so in the first place. but if the missed triangles(about 100 faces) were behind other triangles, then there should no be any holes when viewed it in trimesh or meshlab, but you can see there are holes. am i right ?

[dragonwarrior]

i found the problem that the missing triangles is not caused by rasterization, it's because the index of triangles id in rast_out starts from one, not from zero.

Thank you very much!