Antialiasing limitation

[YYYYYHC]

Thanks for this great work. While I am trying to use it for my project, I found the antialiasing operator somehow limited, and I am here to ask for help.

If I am understanding the documentation correctly, the antialiasing actually works only on two kinds of silhouettes, literally

any edge that connects to just one triangle, or connects two triangles so that one folds behind the other.

Here is my problem: I want to get correct gradience for a third kind of edge: that is a result of two intersected triangles.

Here is a illustration image

it is very clear at this region

for the left edges, the antialiasing works well; but for the right edge, it is aliased.

I am not expecting a direct solution like an existing API (it would be perfect if there is one). I will implement it myself, but I do need some advices/guidance like where would you suggest to modify to make that works.

Best regards & happy new year.

[s-laine]

This is a difficult problem, and there is no support in nvdiffrast to solve it.

One possibility is to split the intersecting geometry using differentiable operations in PyTorch so that in the end there are no intersections. The resulting geometry can then be rendered in such a way that the intersection edge is always counted as a silhouette edge. This can be ensured by, e.g., splitting the vertex indices for the "pieces" you get when resolving the intersections. If the whole process is differentiable, the gradients will propagate to the original vertex positions.

However, making the intersection calculation and remeshing robust enough to work in every case is a major problem in practice, and nearly overlapping meshes can create an unwieldy number of triangles as a result. I would first try everything possible to reformulate the problem in a way that does not require gradients for these kind of intersection edges.

[YYYYYHC]

Thanks for the suggestion!