search

nerfstudio-project / nerfacc

A General NeRF Acceleration Toolbox in PyTorch.
https://www.nerfacc.com/
Other
1.38k stars 112 forks source link

LLFF scenes and NDC coordinates? #120

Open ricshaw opened 1 year ago

ricshaw commented 1 year ago

Does this repo support NDC coordinates for frontal scenes like the LLFF dataset?

liruilong940607 commented 1 year ago

Hi I didn't try but I don't see any reason that nerfacc does not work with NDC space. It is just your rays_o and rays_d need to be in NDC space and then you can just use nerfacc to do marching and rendering as usual.

ricshaw commented 1 year ago

Hey thanks for the reply. I have now implemented the LLFF dataloader with examples/train_mlp_nerf.py, but unfortunately I am getting slightly worse results than vanilla nerf or TensoRF. On the fern scene I get an average PSNR=23.42dB, but the original nerf paper reports 25.17dB and with TensoRF I get 23.91dB. I am using aabb=-1.5,-1.67,-1.0,1.5,1.67,1.0 with render_n_samples=2048 and grid_resolution=128, trained for 100,000 steps. Not quite sure why it is performing worse...

AIBluefisher commented 1 year ago

We can find a discussion here for InstantNGP of how to use the LLFF dataset.

97littleleaf11 commented 1 year ago

@ricshaw could you please share your branch with the LLFF dataloader?

97littleleaf11 commented 1 year ago

Hey thanks for the reply. I have now implemented the LLFF dataloader with examples/train_mlp_nerf.py, but unfortunately I am getting slightly worse results than vanilla nerf or TensoRF. On the fern scene I get an average PSNR=23.42dB, but the original nerf paper reports 25.17dB and with TensoRF I get 23.91dB. I am using aabb=-1.5,-1.67,-1.0,1.5,1.67,1.0 with render_n_samples=2048 and grid_resolution=128, trained for 100,000 steps. Not quite sure why it is performing worse...

The difference between NDC and scene contraction from mipnerf-360 might be the course of worse performance? I also tried to run the LLFF dataset using train_mlp_nerf and can confirm it generates worse results. https://github.com/NVlabs/instant-ngp/discussions/367#discussioncomment-3205524 ngp-pl without NDC also reported a worse result.