search

NVlabs / nvdiffrecmc

Official code for the NeurIPS 2022 paper "Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising".
Other
363 stars 28 forks source link

What is gb_depth exactly? #22

Open half-potato opened 1 year ago

half-potato commented 1 year ago

What exactly are these quantities? I'm trying to add this denoiser to a different method and I don't understand how to adapt my program accordingly. gb_depth appears to contain z0 and |z0-z1|, but I'm not sure what z0 and z1 are. I see they are derived quantities from interpolating triangles on the mesh, but what would these values be for a volumetric renderer using quadrature?

jmunkberg commented 1 year ago

See Appendix B in our paper: https://arxiv.org/pdf/2206.03380.pdf

The depth values are computed here, from the clip space coordinates https://github.com/NVlabs/nvdiffrecmc/blob/main/render/render.py#L228

We adapt the SVGF denoiser, so please refer to that paper for details about the depth guide to the denoiser. That denoiser is designed for surface geometry, and the guides (normals, depth) are assumed to be noise free. It is not trivial to apply these guides to a volumetric renderer, which has a fuzzy depth. See e.g.: https://research.nvidia.com/publication/2021-03_interactive-path-tracing-and-reconstruction-sparse-volumes for a discussion of volume denoising.