search

maturk / dn-splatter

DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing
https://maturk.github.io/dn-splatter/
Apache License 2.0
430 stars 28 forks source link

The backpropagation problem of depth map supervision #73

Open everythoughthelps opened 1 week ago

everythoughthelps commented 1 week ago

I think that if you use the rendered depth as another optimization target besides photometric loss, then you must calculate the gradient from depth. How did you complete the backpropagation part, I can't find it.

maturk commented 1 week ago

Hi @everythoughthelps, thanks for the interest. The gradients and in general the entire backend of Gaussian splatting (forward and backward passes of projection and rasterization) is taken care of within the gsplat dependency. Nerfstudio, and DN-Splatter, are only front-end PyTorch wrappers around main functionalities of gsplat. For more details about gsplat, I wrote a short document outlining the main parts and functionality here.

In a sense, this repo enhances the basic functionality of gsplat with depth sensor, monocular depth, and/or normal information for regularization.

everythoughthelps commented 1 week ago

Thank you!