How do I render a normal map?

[arkadeepnc]

Hi, Thanks for the great code! I was wondering if there is any easy way to render the normal map for a mesh. With the mesh, I can calculate the normal for each triangle, but how do I render it such that each pixel has the color (r,g,b --> u,v,w) corresponding to the world point normal they are imaging?

[pmh47]

Yes; first 'split' the vertices so they are not shared between faces, using dirt.lighting.split_vertices_by_face. Then, vertex_colors passed to rasterise can be exactly the per-vertex normals returned by dirt.lighting.vertex_normals_pre_split. For example, assuming you want world-space normals, something like:

vertices_split, faces_split = dirt.lighting.split_vertices_by_face(vertices_world, faces)
vertices_view_split = tf.einsum('evi,eij->evj', vertices_split, world_to_view_matrices)
projected_vertices_split = tf.einsum('eij,jk->eik', vertices_view_split, projection_matrix)
vertex_normals_split = dirt.lighting.vertex_normals_pre_split(vertices_split, faces_split)
normals = dirt.rasterise_batch(
    background=tf.zeros([episodes_per_batch, frame_height, frame_width, 3]),
    vertices=projected_vertices_split,
    vertex_colors=vertex_normals_split,
    faces=tf.tile(faces_split[None], [episodes_per_batch, 1, 1])
)

[arkadeepnc]

Hi Paul, Thanks for the answer, however, I think there is a minor bug, as you are using 'rasterise_batch' the faces, vertices and, colors need to be tiled to the shape of the background as well. I am writing the fix below.

Also, if anyone is using this code to render only one mesh, only once i.e. 'episodes_per_batch = 1', then the 'e' in 'tf.einsum' needs to be removed or else Tensorflow expects the wrong sized tensor!

Thanks a lot!

vertices_view_split = tf.einsum('evi,eij->evj', vertices_split, world_to_view_matrices)
projected_vertices_split = tf.einsum('eij,jk->eik', vertices_view_split, projection_matrix)
vertex_normals_split = dirt.lighting.vertex_normals_pre_split(vertices_split, faces_split)
normals = dirt.rasterise_batch(
    background=tf.zeros([episodes_per_batch, frame_height, frame_width, 3]),
    vertices=tf.tile(projected_vertices_split[None][episodes_per_batch,1,1]),
    vertex_colors=tf.tile(vertex_normals_split[None],[ episodes_per_batch,1,1]),
    faces=tf.tile(faces_split[None], [episodes_per_batch, 1, 1])
)```

[lll-gen]

@arkadeepnc Have you rendered the normal maps ? and can you share the code? I would appreciate it.

[arkadeepnc]

@Lixz123456 The code I used is exactly as above.