How to sample textureAtlas for texturing function and join meshes into single scene?

[nairb2020]

Hi,

Looking through the documentations it seems very confusing how to use texture atlases where each face has a small patch that needs to be interpolated via barycentric coordinates. It seems that the documentation suggests to concatenate multiple texture maps horizontally, i.e. (32x32) * R (R==5) -> (160x32). However if we do this the bilinear sampling would result in wrong results, since we actually want to interpolated for each barycentric coordinate within the R samples.

I'm used to the implementation of SoftRas style by specifcying (N,F,R,R,D) (N-> batch, F->number of faces, R -> size of patch of this face, D-> RGB). And in Pytorch3D they do bilinear interpolation (currently nearest sample) for each face within RxR patch using barycentric coordinates of pixel. Is there a way to achieve the automatic bilinear sampling of texture Atlases in NVDiffrast?

Thanks so much!

[nairb2020]

2nd related question: Is there a way to join multiple meshes as the same scene and render them with different texture maps (or join texture map) With (N, F, R, R, D) style textureAtlas, it is pretty trivial given that the texture is specified per face. With NVDiffRast, if I want to join 2 meshes with separate texture maps and render them together (with semi-transparency, for example), is there a way to do this?

[s-laine]

There is no proper support for texture atlases at the moment. The (N, F, R, R, D) shape works only for cube maps where F=6, and they need to be accessed via 3-dimensional texture coordinates. At cube edges and corners we interpolate between texels on adjacent faces, so even if you happened to have exactly six textures, you cannot get each texture to wrap individually.

If you have many small texture patches, e.g. one per triangle, the best solution is to place them into a single large texture map and leave sufficient gaps between patches to avoid interpolating across them. Texels in the gaps should be filled with neighboring colors to avoid interpolating to background color near the patch edges. Positioning patches into the atlas is known as texture packing and there are many open-source implementations for that.

Alternatively, if you have just a couple of textures, you can render the meshes individually and composite them afterwards so that in each pixel you pick the closest one based on the (z/w) channel of rasterizer output. This isn't super efficient, though, so perhaps we should consider extending texturing op to handle multiple simultaneous textures.

[nairb2020]

If you guys could provide support for texture atlas it is definitely going to be super helpful!! :) @s-laine

[nairb2020]

If you have many small texture patches, e.g. one per triangle, the best solution is to place them into a single large texture map and leave sufficient gaps between patches to avoid interpolating across them. Texels in the gaps should be filled with neighboring colors to avoid interpolating to background color near the patch edges. Positioning patches into the atlas is known as texture packing and there are many open-source implementations for that.

Regarding this, do you have a good open-source implementation you'd recommend? Also, is leaving sufficient gaps between patches because of MIP mapping so that when NVDiffrast downsamples, it won't corrupt the colors?

Suppose we use 5-level MIP mapping, how much margin do we need? e.g. For 2 32x32 texture maps, is it correct that we need 128x32 texture maps so that we have: 32x32 (texture 1) + 32x32(filled with right edge of texture 1) + 32x32(filled with left edge of texture 2) + 32x32 (texture 2) Or is it OK for the gaps to be smaller?