Unexpected Behavior in Antialiasing Operation

[gcgeng]

Thanks for the great library and your contribution to the community! Recently I was using this library to perform some random inverse rendering tests and encountered some unexpected behavior in dr.antialias() function. Basically what I was trying to do is perform antialiasing on a tensor named accum consisting of all non-negative elements. According to the definition of the antialiasing procedure described in the paper, the antialiased result accum_aa should also be non-negative. (Please correct me if I am wrong here).

However, when I performed this operation, it turns out that accum_aa has one negative element after the operation. Although it may not be a big deal since only one element is negative, I still find this behavior a bit strange to me. So I would like to raise a issue on the possible reason behind this and / or whether this is an known issue or the expected behavior.

The test bench and script to reproduce this issue can be found at here. One may run python test_nvdiffrast.py to reproduce this issue. The result on my machine is:

accum.min() =  tensor(0., device='cuda:0', grad_fn=<MinBackward1>)
accum_aa.min() =  tensor(-0.2310, device='cuda:0', grad_fn=<MinBackward1>)

The environment is as follows:

nvdiffrast.__version__ = 0.3.1
torch.__version__ = 1.13.1
torch.version.cuda = 11.7

Please let me know if you need more information. Thanks in advance!

Cheers, Chen

[s-laine]

Thanks for bringing up this interesting case. The coverage estimation in the antialiasing op is only approximate, which leads to a rather surprising result in this scenario. The test case has a silhouette pixel that is surrounded by the background from three sides in a way that all three pixel-to-background transitions occur over valid silhouette edges.

Each of these three transitions blend some of the background value into the pixel value, and the overall adjustment happens to be greater than the original value in the pixel. The setup looks something like the sketch below, such that the red, green, and blue areas sum up to more than the area of the pixel itself (overlapping areas count as double). Thus the result is not an interpolation but extrapolation between the original pixel value and the background value.

With a single silhouette edge affecting a pixel, the adjustment is at most 50% of the original value, and with two edges at most 100% which is still on the safe side, but with three edges the total can go up to 150%.

Unfortunately there is no easy way to fix this in the antialiasing op, as all the adjustments are made separately and in parallel — the total adjustment factor is never explicitly determined, so there is no place where it could be clamped to one.

Perhaps there's a way to clamp the output somehow in your application in a way that avoids the problem.

[gcgeng]

Thank you for your detailed explanation! This answers my question. I think in practice this will not cause a lot of problems since it occurs very sparsely and we can always clip the output. So I will close this issue.