Rendered Normal Map

[YuhsiHu]

Hi, thank you for your great work!

When I tried to use your code to generate normal maps for the input, I got a tensor that always like (0, 0, 1) for every pixel. This is strange. Here is my code:

from diff_gauss import GaussianRasterizationSettings, GaussianRasterizer

raster_settings = GaussianRasterizationSettings(
        image_height=int(data['height'][idx]),
        image_width=int(data['width'][idx]),
        tanfovx=tanfovx,
        tanfovy=tanfovy,
        bg=bg_color,
        scale_modifier=1.0,
        viewmatrix=data['world_view_transform'][idx],
        projmatrix=data['full_proj_transform'][idx],
        sh_degree=3,
        campos=data['camera_center'][idx],
        prefiltered=False,
        debug=False
    )

rasterizer = GaussianRasterizer(raster_settings=raster_settings)
rendered_image, rendered_depth, rendered_normal, rendered_alpha, radii, extra = rasterizer(
        means3D = pts_xyz,
        means2D = screenspace_points,
        shs = None,
        colors_precomp = pts_rgb,
        opacities = opacity,
        scales = scales,
        rotations = rotations,
        cov3Ds_precomp = None,
        extra_attrs = None
    )

I visualized the estimated normal maps which come from depth maps, ground truth normal maps, and your rendered normal maps for example.

Is there anything that I need to do to get the correct normal maps? Thank you for your time.

[slothfulxtx]

Hi, @YuhsiHu Based on my initial observation, you'd better check the visualization code of your normal map.

[YuhsiHu]

Hi, @slothfulxtx . I use the same visualization code for all these tensors. And here is the code:

    def save_normal_map(self, normal_map_tensor, filename):
        # Step 1: Remove the batch dimension [B=1, H, W, 3] -> [H, W, 3]
        normal_map = normal_map_tensor.squeeze(0).detach().cpu().numpy()  # Convert to NumPy

        # Step 2: Rescale normal map from [-1, 1] to [0, 255] for visualization
        normal_map = (normal_map + 1) / 2.0 * 255.0
        normal_map = normal_map.astype(np.uint8)  # Convert to 8-bit integer

        # Step 3: Create and save the image
        img = Image.fromarray(normal_map)
        img.save(filename)

I also directly printed the rendered_normal, but it is always like (0, 0, 1) for every pixel...

[slothfulxtx]

quite aweird, i don't find any bugs in your visualization scripts. do you obtain a reasonable depth map?

[YuhsiHu]

Yes, I am using MVSGaussian. The depth map from MVS is reasonable, but after renderer, the depth map and normal are strange.

[slothfulxtx]

Can you save the values of all arguments fed into diff-gauss-rasterizer as a Npz file and share it with me? Maybe something wrong exists in the data.