simple question about pixel coordinate rounding problem when rendering triangles using `rasterize()`

[Karbo123]

Frist, I really appreciate the release of this great work, it makes rasterization more efficient. However, recently I want to figure out the internal pixel coordinate rounding mechanism in nvdiffrast, but I cannot understand how it treat the pixel coordinate.

The following codes try to render two triangles which I think should look like:

import imageio
import numpy as np
import torch
import nvdiffrast.torch as dr
import sys

def tensor(*args, **kwargs):
    return torch.tensor(*args, device='cuda', **kwargs)

glctx = dr.RasterizeCudaContext()

res = 8
x = 1 - 1 / res
pos = tensor([[[-x, -x, 0, 1], [x, -x, 0, 1], [-x, x, 0, 1], [x, x, 0, 1]]], dtype=torch.float32)
col = tensor([[[1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 1, 1]]], dtype=torch.float32)
tri = tensor([[0, 1, 2], [2, 1, 3]], dtype=torch.int32)

rast, _ = dr.rasterize(glctx, pos, tri, resolution=[res, res])
out, _ = dr.interpolate(col, rast, tri)

img = out.cpu().numpy()[0, ::-1, :, :] # Flip vertically.
img = np.clip(np.rint(img * 255), 0, 255).astype(np.uint8) # Quantize to np.uint8

print("Saving to 'tri.png'.")
imageio.imsave('tri.png', img)

I think the two triangles should cover the whole image space (i.e. each pixel should have its corresponding triangle), but the rendering image actually looks like this:

You can find there are some black pixels at the right/top borders, which indicates there are not any triangles projecting to these locations.

I think the triangles should be projected to these coordinates, and I expect the rendering result doesn't contain any black pixels.

Could you elaborate mroe on that? Thank you very much!

[Karbo123]

But if I change pos to:

e = 0.01
pos = tensor([[[-x, -x, 0, 1], [x+e, -x, 0, 1], [-x, x+e, 0, 1], [x+e, x+e, 0, 1]]], dtype=torch.float32)

the rendering result becomes:

notice that the three pixels at the corners are not 255 (i.e. there are some other values such as 254 and 252)

If I want to make all the pixels at the corners become either 0 or 255, what should I change the codes?

[s-laine]

Your first illustration with triangles and pixels is correct. The reason that some pixels are not filled is that on the sides the geometry does not overlap pixel centers but goes exactly through them. In this situation, which pixels are treated as covered follows so-called rasterization rules. Here is an example for DirectX where the vertical direction works the opposite of OpenGL, but the idea is the same.

Getting the result you're after is not quite trivial. Probably the best solution is to extend your geometry so that it properly overlaps the pixels, e.g., by setting x = 1 so that the triangles reach the corners of the image, and adjust the colors in the opposite direction so that their values at centers of corner pixels are correct. Here is an example:

x = 1
e = 0.5 / (res - 1)
pos = tensor([[[-x, -x, 0, 1], [x, -x, 0, 1], [-x, x, 0, 1], [x, x, 0, 1]]], dtype=torch.float32)
col = tensor([[[1+2*e, -e, -e], [0, 1+e, -e], [0, -e, 1+e], [1+2*e, 1+e, 1+e]]], dtype=torch.float32)

This will give the exact colors (up to rounding errors) at pixel centers at the corner pixels of the image.

[Karbo123]

Thanks for the solution!