search

NVlabs / nvdiffrec

Official code for the CVPR 2022 (oral) paper "Extracting Triangular 3D Models, Materials, and Lighting From Images".
Other
2.1k stars 222 forks source link

Implementations of MLPTexture3D #5

Closed bennyguo closed 2 years ago

bennyguo commented 2 years ago

Hello,

Thanks for releasing the code for this amazing work!

I've been going through the code and find some implementations in TextureMLP3D confusing: 1) What's the meaning of scaling the gradient? I set gradient_scaling=1.0 and did not find much difference in the output. 2) Why not use the tcnn MLP but only the tcnn Encoding?

Thanks!

jmunkberg commented 2 years ago

Thanks!

  1. The gradient scaling makes the hash grid encoder train to a sharper result in the first pass, most notably on models with high frequency textures. It is not a crucial optimization however, as the second pass, when we switch to standard 2D textures, usually increases the sharpness regardless.

  2. We use only the encoder from tcnn for compatibility reasons. The cutlass MLP:s were not compatible with some older servers we used in the development, so with a vanilla PyTorch MLP, we could deploy the code on more machines. It is very easy to switch back to the tcnn MLP (or preferably, the combined encoder+MLP from tcnn) if you want.

bennyguo commented 2 years ago

Thanks for the reply! Closing the issue now.