This is the official implementation of VaxNeRF (Voxel-Accelearated NeRF).
VaxNeRF provides very fast training and slightly higher scores compared to original (Jax)NeRF!!
Visual Hull (1sec) | |||
NeRF (10min) | VaxNeRF (10min) | Vax-MipNeRF (10min) |
Please see the README of JaxNeRF.
The jax and jaxlib versions that we have tested are as follows.
jax 0.2.24
jaxlib 0.1.69+cuda111
jax 0.2.17
jaxlib 0.1.65+cuda110
# make a bounding volume voxel using Visual Hull
python visualhull.py \
--config configs/demo \
--data_dir data/nerf_synthetic/lego \
--voxel_dir data/voxel_dil7/lego \
--dilation 7 \
--thresh 1. \
--alpha_bkgd
# train VaxNeRF
python train.py \
--config configs/demo \
--data_dir data/nerf_synthetic/lego \
--voxel_dir data/voxel_dil7/lego \
--train_dir logs/lego_vax_c800 \
--num_coarse_samples 800 \
--render_every 2500
python eval.py \
--config configs/demo \
--data_dir data/nerf_synthetic/lego \
--voxel_dir data/voxel_dil7/lego \
--train_dir logs/lego_vax_c800 \
--num_coarse_samples 800
Original NeRF
python train.py \
--config configs/demo \
--data_dir data/nerf_synthetic/lego \
--train_dir logs/lego_c64f128 \
--num_coarse_samples 64 \
--num_fine_samples 128 \
--render_every 2500
VaxNeRF with hierarchical sampling
# small `num_xx_samples` needs more dilated voxel (see our paper)
python visualhull.py \
--config configs/demo \
--data_dir data/nerf_synthetic/lego \
--voxel_dir data/voxel_dil47/lego \
--dilation 47 \
--thresh 1. \
--alpha_bkgd
# train VaxNeRF
python train.py \
--config configs/demo \
--data_dir data/nerf_synthetic/lego \
--voxel_dir data/voxel_dil47/lego \
--train_dir logs/lego_vax_c64f128 \
--num_coarse_samples 64 \
--num_fine_samples 128 \
--render_every 2500
Visual Hull
--dilation 11
/ --dilation 51
for NSVF-Synthetic dataset for training VaxNeRF without / with hierarchical sampling.Lifestyle
, Spaceship
, Steamtrain
scenes (included in the NSVF dataset) do not have alpha channel, please use following options and remove --alpha_bkgd
option.
--thresh 0.95
, Spaceship: --thresh 0.9
, Steamtrain: --thresh 0.95
NeRFs
--small_lr_at_first
option for original NeRF training on the Robot
and Spaceship
scenes to avoid local minimum. jaxnerf
branch) and VaxNeRF (vaxnerf
branch) heremain
branch (derived from the vaxnerf
branch) contains the following features.
Please use the following bibtex for citations:
@article{kondo2021vaxnerf,
title={VaxNeRF: Revisiting the Classic for Voxel-Accelerated Neural Radiance Field},
author={Kondo, Naruya and Ikeda, Yuya and Tagliasacchi, Andrea and Matsuo, Yutaka and Ochiai, Yoichi and Gu, Shixiang Shane},
journal={arXiv preprint arXiv:2111.13112},
year={2021}
}
and also cite the original NeRF paper and JaxNeRF implementation:
@inproceedings{mildenhall2020nerf,
title={NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis},
author={Ben Mildenhall and Pratul P. Srinivasan and Matthew Tancik and Jonathan T. Barron and Ravi Ramamoorthi and Ren Ng},
year={2020},
booktitle={ECCV},
}
@software{jaxnerf2020github,
author = {Boyang Deng and Jonathan T. Barron and Pratul P. Srinivasan},
title = {{JaxNeRF}: an efficient {JAX} implementation of {NeRF}},
url = {https://github.com/google-research/google-research/tree/master/jaxnerf},
version = {0.0},
year = {2020},
}
We'd like to express deep thanks to the inventors of NeRF and JaxNeRF.
Have a good VaxNeRF'ed life!