NVlabs / nvdiffrast

Nvdiffrast - Modular Primitives for High-Performance Differentiable Rendering
Other
1.31k stars 139 forks source link

How to use the API to interpolate attributes defined on face? #71

Closed 07hyx06 closed 2 years ago

07hyx06 commented 2 years ago

Hi! Thanks for your code.

I want to interpolate attributes on the mipmap stack texture, but the texture coordinate is defined on the face, not on the vertex. Each vertex may have several corresponding texture coordinates. E.g., in SMPL, the texture coordinate is defined on each face, as shown in the figure below.

图片1

I think if the texture coordinate is defined on the vertex, the solution is straightforward: call rasterize, then interpolate, finally texture. But in this case, how can I use the API to implement it (especially the image-space derivatives of attributes)?

s-laine commented 2 years ago

You can use a separate index tensor for the interpolation op. The indices it contains have to be consistent with the supplied attribute tensor, but they can be completely separate from vertex position indices used in rasterization. However, the triangles and vertices must be in the same order as in the index tensor used in the rasterization op.

The "earth" example uses this technique because the mesh has face-specific texture coordinates at texture seams. There is a vtx_pos tensor containing vertex positions and corresponding pos_idx that points to them, used in the rasterization op. The uv coordinates in vtx_uv are referenced in uv_idx tensor, used in interpolation. pos_idx and uv_idx have the same shape and logically their contents refer to the same faces and vertices.

Because attribute pixel derivatives are computed separately in each pixel, they will work automatically.

07hyx06 commented 2 years ago

Got it, thanks!