search

nerfstudio-project / nerfstudio

A collaboration friendly studio for NeRFs
https://docs.nerf.studio
Apache License 2.0
8.87k stars 1.18k forks source link

@torch_compile() for function get_viewmat() cause the bad training results. #3242

Open Miaosheng1 opened 1 week ago

Miaosheng1 commented 1 week ago

Describe the bug I've been testing SplatFacto on my device and encountered a peculiar issue. When I follow the original code and apply @torch_compile(), my results end up in local minima and fail to achieve good outcomes. However, when I comment out @torch_compile(), my code works correctly. I'm not sure why this happens, but I suggest avoiding the use of this decorator for speed optimization until it becomes more stable.

https://github.com/nerfstudio-project/nerfstudio/blob/9b3cbc79bf239eb3c69e7c288632aab02c4f0bb1/nerfstudio/models/splatfacto.py#L99-L115

Training PSNR ( add the decorator ): image

Training PSNR (Not add the decorator ): image

My running environment is as follows:

maturk commented 1 week ago

torch version 2.0.1 is known to have issues. Can you upgrade to torch 2.1.2+. See #3235, #3230