google-research / multinerf

A Code Release for Mip-NeRF 360, Ref-NeRF, and RawNeRF
Apache License 2.0
3.57k stars 338 forks source link

Training and Testing views behave differently #80

Closed donjiaking closed 1 year ago

donjiaking commented 1 year ago

I use custom dataset which provides correct poses, blender_256.gin as configuration file, and trained for 400000 iterations. The training views looks normal, which achieves psnr of approximately 30. But the test view renderings are completely blury (an example is shown below), psnr only around 16. I really wonder what could possibly cause this problem. I checked the dataset has no problem. Below is my configuration. Thank you.

Config.dataset_loader = 'blender'
Config.batching = 'single_image'
Config.near = 0.1
Config.far = 20
Config.eval_render_interval = 5
Config.data_loss_type = 'mse'
Config.adam_eps = 1e-8

Config.batch_size = 2048
Config.max_steps = 500000
Config.lr_init = 0.0005
Config.factor = 4
Config.render_chunk_size = 2048

...

image

jonbarron commented 1 year ago

I'd guess the problem is in your camera poses. If the poses are wrong or in a different format than our code expects, NeRF can get very high train PSNR by "painting" each image right in front of each camera, but it will generalize poorly.