more than 2^24 triangles

[za-cheng]

Hi,

I really enjoyed using this code base and now am building a 3D reconstruction pipeline on it.

One issue I had is that I need to rasterise an entire volumetric grid for not just the nearest but every triangle inside. I'm using a DepthPeeler for this but the returned triangle ID is stored in 32-bit float point format so there's a precision overflow when number of triangles exceeds 2^24 (in my case it could go up to billions)

I understand nvdiffrast is probably not built for rasterising that many triangles but is there any chance you would support this in future or is there any quick fix I can do?

Appreciate your time. Cheers.

[s-laine]

There is an implemented but yet unreleased patch that helps the situation a bit by allowing more than 2^24 triangle IDs to be represented in the rasterizer output. However, it would probably not be sufficient for your needs, as billions of triangles will not work with a 32-bit value, and the patch doesn't even allow the use of every possible 32-bit value.

Also note that the depth peeler is approximate, and you are not guaranteed to rasterize every triangle: only one triangle per projected depth value will be rasterized, so you will miss some if the depths are packed densely. Finally, rasterizing extremely high depth complexity is very inefficient, because every depth peeler pass will process all depth layers even though only one ends up in the result image.

I suppose you could rasterize the grid one layer at a time, in which case you wouldn't need to rely on the depth peeler. You can determine the best layer orientation based on the camera position relative to the volume, so that you never rasterize the layers "edge-on". This way you'd only need to consider triangle IDs worth of one grid layer.

Alternatively, your use case might be better suited for a ray marcher of some sort, in case your data is originally a 3D volume (such as a signed distance field) instead of a mesh. If you construct 3D sampling points corresponding to intersections of rays at centers of pixels and the depth layers, you could use torch.nn.functional.grid_sample() to sample the volume.

[za-cheng]

Thank you for the prompt reply and for the advice! I did not know the depth peeler is approximated - that actually explains some problems I'm getting.

My grid is actually irregular and behaves more like a densely packed triangle soup (with less than 2^32 triangles) where triangles don't overlap or intersect, and are only connected on edges or at vertices. I don't see an easy way to rasterise by layer but perhaps I should try ray marching libraries like Embree as you suggested.

Anyways thanks again for the suggestions.