half-potato / nmf

Our method takes as input a collection of images (100 in our experiments) with known cameras, and outputs the volumetric density and normals, materials (BRDFs), and far-field illumination (environment map) of the scene.
https://half-potato.gitlab.io/posts/nmf/
MIT License
53 stars 3 forks source link

[Question] About the derivation of normal vector from the density field #4

Closed costrice closed 1 year ago

costrice commented 1 year ago

In the released paper Neural Microfacet Fields for Inverse Rendering (ICCV'23), it was mentioned in Section C of the appendix that

To calculate the normal vectors of the density field, we apply a finite difference kernel, convolved with a 3×3 Gaussian smoothing kernel with σ = 1, then linearly interpolate between samples to get the resulting gradient in the 3D volume.

I appreciate this idea to use smoothed gradient as the shading normal, and want to check the code that implements it. However, I failed to find the code that implements this finite difference and gaussian filtering functionality.

The forward method of the TensorNeRF class in modules/tensor_nerf.py seems to be using the compute_normals method of the TensorBase class, but this method directly use analytically derived gradients, rather than gradients from finite difference, without doing Gaussian filtering.

Then, I searched the project for 'normal' and found another function called compute_density_norm in fields/triplanar.py that seems doing finite difference (I'm not sure). But this class seems to be not in active use.

I would be very grateful if the authors could refer me to the code implementing the normal derivation described in the paper. Thanks in advance!

half-potato commented 1 year ago

The gradient calculation can be found here: modules/grid_sample_Cinf.py

This special grid sample operator is used here: fields/tensoRF.py

costrice commented 1 year ago

Thanks very much for your immediate and helpful reply!