RasterizeCudaContext behaves very differently from RasterizeGLContext

[XCR16729438]

I tested Deep3DFaceRecon_pytorch with both RasterizeGLContext(as default in this project) and RasterizeCudaContext, and they behaved very differently.

The key part of code is in Deep3DFaceRecon_pytorch/util/nvdiffrast.py. You can change rasterizer in line 58. You may change line 78 from mask = (rast_out[..., 3] > 0).float().unsqueeze(1) to mask = (rast_out[..., 3] > float("-inf")).float().unsqueeze(1) to ignore the background to show the result clearly.

I got complete rendered human face with RasterizeGLContext, but broken image with RasterizeCudaContext. I tested both in Linux and Windows with various versions of torch, I believe the problem is irrelevant with platform and enviromnent.

Result of RasterizeGLContext: Result of RasterizeCudaContext:

[s-laine]

Thanks for the report that made it easy to reproduce the issue. There was a bug in a test that detects when a tiny triangle doesn't hit any pixels so that it can be discarded early. Fixed now in the latest release - closing.

[cleardusk]

Based on this commit https://github.com/NVlabs/nvdiffrast/commit/fad71a4ad3dc8530a4eeaeb55a65b2e8c127d69d, if enabling the back face culling, e.g., cr->setRenderModeFlags(RenderModeFlag_EnableBackfaceCulling);, the visibilities of pixel to triangles seems not right. The tiny triangles will be regarded as back faces.

[s-laine]

Is this something you have seen when running the rasterizer? The fix in the commit ensures that the tiny triangle culling test functions the same way regardless of triangle facing, as previously it didn't work correctly on backfacing triangles. Or that at least is the intention. If you have a test case that doesn't work as expected, please let me know.

[cleardusk]

Is this something you have seen when running the rasterizer? The fix in the commit ensures that the tiny triangle culling test functions the same way regardless of triangle facing, as previously it didn't work correctly on backfacing triangles. Or that at least is the intention. If you have a test case that doesn't work as expected, please let me know.

I will report a case tomorrow.

[cleardusk]

Using bfm as the input: bfm.zip. An obj file lies in this zip file, set the yaw to 135°.

• Without the backface culling

• With the backface culling

I enable the backface culling in torch_rasterize.cpp, by adding a line cr->setRenderModeFlags(CR::CudaRaster::RenderModeFlag_EnableBackfaceCulling); // hack here in Line99.

[s-laine]

I believe this is the correct output, and the problem is in what you expect the flag to do.

Backface culling means that triangles with certain 2D winding order after projection are not rendered. It is an optimization often used in games, and can only be used if your meshes are closed, have no holes, and have consistent triangle winding in 3D (the geometry is two-manifold as they say). The idea is that you can skip rendering backfacing triangles and it doesn't affect the output image if these conditions are met.

Since many meshes don't have those properties (e.g., face meshes often have holes and don't enclose a 3D volume), backface culling is not used in nvdiffrast. The Cuda rasterizer codebase I adapted to nvdiffrast has that option, and I actually considered removing it altogether as it's not needed. You are not expected to enable it, but I suppose you can if you know your meshes are such that the above conditions are met. The performance gains will almost certainly be immeasurably small, though.

To reiterate, you are not supposed to use backface culling to influence the output image — it's strictly a performance optimization. For example, the silhouette edge detection in antialiasing op will not work properly if backface culling changes the rendered image.

[cleardusk]

Thanks for the detailed reply, I got the design of nvdiffrast about the back-face culling. Actually, the performance is not my point. I think the rasterization can support the back-face culling (if not considering the compatibility of the antialiasing op, etc.) with small modifications. I will give it a try during my spare time.