search

chenhsuanlin / bundle-adjusting-NeRF

BARF: Bundle-Adjusting Neural Radiance Fields 🤮 (ICCV 2021 oral)
MIT License
793 stars 114 forks source link

graph.pose_noise not saved for models with blender dataset? #29

Closed geoffreymantel closed 2 years ago

geoffreymantel commented 2 years ago

I hope this is not too dumb of a question, but I'm having trouble understanding how evaluate.py can correctly derive the mean rotation and translation errors from a model checkpoint file if the camera noise is not saved as part of the checkpoint?

As I understand it, self.graph.se3_refine is learning pose corrections from the identity (for non-blender models) or from randomly-initialized noise (for blender models). In the blender case, when evaluating a trained model, wouldn't the camera noise be different between training and evaluation?

Thanks for your help!

chenhsuanlin commented 2 years ago

Hi @geoffreymantel, the generated noises would be the same under the same fixed random seed (set here). This allows us to explore BARF's behavior more easily, but pre-loading from a file is probably also a good solution. Hope this helps!

geoffreymantel commented 2 years ago

Ah! That's what I started to suspect. I've updated my own experiment to save the pose_noise to the checkpoint - I've been running experiments with a batch job tool (ClearML) which somehow made the train and evaluate initialize the pose to different values. Thanks for the info!