Depth peeling + anti-aliasing: background bleeds into the foreground

[nicolas-guichard]

In render.py, you composite the scene back to front, anti-aliasing each step of the way:

# Composite BACK-TO-FRONT
for color, rast in reversed(layers):
    alpha     = (rast[..., -1:] > 0) * color[..., 3:4]
    accum_col = torch.lerp(accum_col, color[..., 0:3], alpha)
    if antialias:
        accum_col = dr.antialias(accum_col.contiguous(), rast, v_pos_clip, mesh.t_pos_idx.int()) # TODO: need to support bfloat16

This can cause the background to bleed though in some cases when rendering transparent parts.

For instance, in the attached scene of a rotated transparent box in from of an opaque plane, we can get the background to appear in front of the opaque plane:

Layer	Image
Background
Last layer: plane behind the box
Accum
Middle layer: backface of the box
Accum
Front layer: frontface of the box + remaining of the plane
Final image: bits of the background made it through the layers

Those artifacts also impact the resulting gradients obviously.

[JHnvidia]

Hi @Nico264,

Thanks for reporting this. Unfortunately there's no easy way of working around the issue. Nvdiffrast wasn't originally designed with depth peeling in mind, and we are abusing it by combining peeling and the anti aliasing primitives.

The issue in this case is that the distant plane is split into two layers (first and last in your images). The antialias function only operates on a single layer, and therefore thinks there's a silhouette in the background layer, even though the plane continues in the foreground layer. There are some workarounds, but solving the problem correctly would require bigger changes to the renderer:

• Only use antialiasing for the first layer. This will disable the geometry (silhouette) gradient term for geometry that is not directly visible, but could work in some cases.
• Switch to depth-bucketing (e.g. fixed depth ranges go into separate buffers). This solves the silhouette issue, but cant handle transparency within a bucket.
• Augment the nvdiffrast anialiasing primitive to understand multiple layers, and reomove false silhouettes.

[nicolas-guichard]

Hi @JHnvidia and thanks for your answer.

I was indeed under the impression that this was not trivially fixable. I went for a variant of your first suggestion: I repeatedly apply anti-aliasing on the final composition according to each layer's rast, ignoring fragments that are occluded or fully transparent.

The result still isn't really correct but the artifacts are less obvious and I still get some gradients for silhouettes behind transparent objects.

I guess you can close this as won't fix/known issue if fixing this is out of scope.