search

NVlabs / nvdiffrast

Nvdiffrast - Modular Primitives for High-Performance Differentiable Rendering
Other
1.37k stars 146 forks source link

Is it possible to get visible vertices of a mesh by rasterizion? #55

Closed zoe2718 closed 2 years ago

zoe2718 commented 2 years ago

The 2d visible mask can be obtained by using the output of nvdiffrast.torch.rasterize: rast_out[..., 3] > 0. Is it possible to know which vertices are visible during the rasterizion? Or is there any other way to get visible vertices if I use nvdiffrast for rendering.

s-laine commented 2 years ago

You can get an approximate answer by collecting the vertex indices of all triangles visible in the image, whose IDs (offset by 1) are available in rast_out[..., 3]. However, if a triangle is small enough so that it falls between the pixel centers, it will not be present in the output buffer, and so you may miss some of the vertices that would actually be visible to the camera. You'd need every triangle connected to a vertex to be missing from the output in order to miss the vertex altogether, so this helps the situation somewhat.

Increasing the rendering resolution makes it more likely that visible triangles will be present in the output image. If your application can cope with an approximate answer, and your mesh isn't overly detailed, you can probably find a resolution where the result is good enough for your purposes.

zoe2718 commented 2 years ago

Thanks for the quick response. That solved my problem.