search

POSTECH-CVLab / SCNeRF

[ICCV21] Self-Calibrating Neural Radiance Fields
MIT License
463 stars 45 forks source link

How dose "multiplicative_noise" infrence results? #9

Closed DRosemei closed 2 years ago

DRosemei commented 2 years ago

Thank for your great works! I want to know how "multiplicative_noise" infrences results. Will results be better if I use add it for training? I find that you set it to be "True" in all experiments. Thanks in advance! Reference codes are here: https://github.com/POSTECH-CVLab/SCNeRF/blob/master/model/camera_model.py#L166

DRosemei commented 2 years ago

By the way, in nerf--，it uses axis-angle of 3 dimentions to optimize rotation. While in your work, you use "orth" of 6 dimentions. I want to know what "orth" means, and is there any difference between axis-angle and "orth"?

jeongyw12382 commented 2 years ago

At first, when we do not set the multiplicative noise, it easily converges to the wrong camera information. One prior we've used here is that the errors wouldn't be so large. However, when you use 360-scenes, It might be helpful to remove the multiplicative noise.

jeongyw12382 commented 2 years ago

For the second question, I strongly recommend taking a look at the NeurIPS paper that first proposes continuous rotation representation. This paper introduces 6-dimensional representation which is beneficial for Neural Nets to learn from. https://openaccess.thecvf.com/content_CVPR_2019/papers/Zhou_On_the_Continuity_of_Rotation_Representations_in_Neural_Networks_CVPR_2019_paper.pdf

jeongyw12382 commented 2 years ago

Mail(jyw123822@gmail.com) me if you have further questions about the ideas here. I could spend time discussing the relevant ideas using zoom.

jeongyw12382 commented 2 years ago

Closing the issue after the response.

DRosemei commented 2 years ago

Thanks for your great help and I'll try it!