daniilidis-group / neural_renderer

A PyTorch port of the Neural 3D Mesh Renderer
Other
1.12k stars 248 forks source link

Confuse about the 5-dims texture size. #75

Open Arthur151 opened 4 years ago

Arthur151 commented 4 years ago

I really appreciate your great work! But I am confusing about the dimension of texture... So why the texture has 5 dimensions, let see: face size texture size texture size texture size 3 = 7576 4 4 4 3 I am confused about the usage of texture size. Is it supersampling that devides the pixel into 4 parts? Or something else? Looking forward to your reply. Best wishes.

lvvvmd commented 4 years ago

let's take texture size=4 as an example, so we have 4 4 4=64 items for each face which is save in the texture matrix with shape 4 4 4. It means each pixel's color is sampled from the 64 candidate colors. And each triangle face has 3 vertices(A B C), and each vertices can be regarded as a vector(A, B, C) from the origin to each vertices. And any point P in this triangle surface can be represented as a linear combination of these 3 vector: P=xA+yB+zC because we are in a 3D coordinate-system and x+y+z=1 . And then the color of P is sampled from the texture matrix: tex[int(x 4)-1][int(y 4)-1][int(z * 4)-1] . And here you can replace 4 by any other texture size as you want and bigger texture size means more candidate color for each face.

Arthur151 commented 4 years ago

Thanks for your kind reply! So if there are dense vertex, a lager texture metric may lead to a better rendering result. But I still have a question. The form of linear equation P=xA+yB+zC is similar to the derivation of the barycentric coordinates. The only difference is that when deriving the barycentric coordinates, z=1. In my view, the texture is to store the color bank for sampling the color of each face. Is there any meanings to derive the z value? To sampling the color off the triangle plane? So I think that 2 dimension (44, 1616 or even 32*32)is enough for super-sampling more subtle colors. Why there is a z dim?