search

chenhsuanlin / bundle-adjusting-NeRF

BARF: Bundle-Adjusting Neural Radiance Fields 🤮 (ICCV 2021 oral)
MIT License
793 stars 114 forks source link

Image from validation cameras 'moves away' #13

Closed szymanowiczs closed 2 years ago

szymanowiczs commented 2 years ago

Hi Chen-Hsuan,

Thanks for the great work. Every time the validation code is run, the resulting renders seem to move away from the camera (see images). MicrosoftTeams-image (3) MicrosoftTeams-image (4) MicrosoftTeams-image (5) MicrosoftTeams-image (6)

The underlying reason is that the arguments passed to preprocess_camera.py are passed by reference, so preprocess_camera.py changes the intrinsics of the underlying camera. This change occurs every time the validation code is run, so the intrinsics change, and the validation images become smaller and smaller. To avoid this issue, I have created a PR that detaches and clones the pose before preprocessing, therefore passing a copy of the intrinsics to be modified appropriately. https://github.com/chenhsuanlin/bundle-adjusting-NeRF/pull/12

Let me know if we can pull this into main - it avoids the objects 'moving away', making the validation images easier to analyse.

Kind regards, Stan

chenhsuanlin commented 2 years ago

Thanks @szymanowiczs for the great catch! I've modified your PR to clone the variables (no need to detach) in preprocess_camera() instead -- ideally variables shouldn't be modified in-place here.