pierremerriaux-leddartech
commented
5 months ago Hi few more experiments on my side:
I used camera poses directly got from pandaset:
we got this reconstruction:
I added gaussian noise on trainset camera poses:
- translations : sigma .2m (in real world coordinate, before rescale in -1 1 cube)
- rotation : sigma 1degree
of course the reconstruction is very bad like that:
When I activate camera pose optimization I get this one:
It difficult to compare metric because even if I didn't add noise to eval set, the "world" seems to be not reconstructed exactly at the same position. So eval metrics are badly similar for noisy and optimized poses. But we could have a look at train psnr (noisy_notInEval2, is the optimize one):
By carefully manually review images between no noise and optimized poses version, optimized is a little less quality. But in another hand I now that in my in case, even with no noise, when I render on shift trajectory the quality decrease.
And at the end, camera optimization don't help me for multiview cameras (front, left, right....), at it was in case nerffacto https://github.com/nerfstudio-project/nerfstudio/issues/2863
Hi @vye16 and @kerrj In order to re activate camera pose opt in nerfstudio, I cherry pick the gradient backward proposed in branch : https://github.com/nerfstudio-project/gsplat/tree/vickie/camera-grads
My tests were not really good. Did you get any plan to continue with this feature ? Anyway I could help on it ? Any idea how we could test properly. thanks