search

wanmeihuali / taichi_3d_gaussian_splatting

An unofficial implementation of paper 3D Gaussian Splatting for Real-Time Radiance Field Rendering by taichi lang.
Apache License 2.0
637 stars 58 forks source link

Nans in camera gradient #141

Closed cameronosmith closed 9 months ago

cameronosmith commented 9 months ago

Hi! Thanks for the awesome project! I found that there are sometimes nans or infs in the camera pose gradients (around 1/10 iterations). I'm just wondering whether you experienced these too? Also I'm new to Taichi but doesn't it support automatic differentiation? If so why do you write the gradients by hand? Thanks again!

wanmeihuali commented 9 months ago

Hi @cameronosmith , I also noticed the NaN issue, and I'm still debugging it... I guess the most simple way to get rid of it at this time is filtering them by hook.

For taichi auto-dff, taichi does support auto-diff, I think the stable version still does not support auto-diff for pytorch tensor, but the nightly version already does. In this project, one of the loops in backward function shall be able to be handled by auto-diff(from grad alpha to grad xyz/q). However, the feature is buggy at least when I tried it several months ago, I once had a PR, and the calculation result was wrong, most likely because race condition in reducing gradient. I think they have some updates recently, but I don't have time to try it.