Project Page | Arxiv | Data | Pretrained Model
Wenjing Bian, Zirui Wang, Kejie Li, Jiawag Bian, Victor Adrian Prisacariu. (CVPR 2023 highlight)
Active Vision Lab, University of Oxford.
git clone https://github.com/ActiveVisionLab/nope-nerf.git
cd nope-nerf
conda env create -f environment.yaml
conda activate nope-nerf
Tanks and Temples:
Our pre-processed Tanks and Temples data contains the 8 scenes shown in the paper. Each scene contains images, monocular depth estimations from DPT and COLMAP poses. You can download and unzip it to data
directory.
NeRF LLFF:
We also provide config file for NeRF LLFF dataset. You can download the dataset and unzip it to data
directory. One example of the config file is configs/LLFF/fern.yaml
.
If you want to use your own image sequence with customised camera intrinsics, you need to add an intrinsics.npz
file to the scene directory. One example of the config file is configs/Test/images.yaml
(please add your own data to the data/Test/images
directory).
Monocular depth map generation: you can first download the pre-trained DPT model from this link provided by Vision Transformers for Dense Prediction to DPT
directory, then run
python preprocess/dpt_depth.py configs/preprocess.yaml
to generate monocular depth maps. You need to modify the cfg['dataloading']['path']
and cfg['dataloading']['scene']
in configs/preprocess.yaml
to your own image sequence.
python train.py configs/Tanks/Ignatius.yaml
where you can replace configs/Tanks/Ignatius.yaml
with other config files.
You can monitor on http://localhost:6006 the training process using tensorboard:
tensorboard --logdir ./out --port 6006
For available training options, please take a look at configs/default.yaml
.
Evaluate image quality and depth:
python evaluation/eval.py configs/Tanks/Ignatius.yaml
To evaluate depth: add --depth
. Note that you need to add ground truth depth maps by yourself.
Evaluate poses:
python evaluation/eval_poses.py configs/Tanks/Ignatius.yaml
To visualise estimated & ground truth trajectories: add --vis
Novel view synthesis
python vis/render.py configs/Tanks/Ignatius.yaml
Pose visualisation (estimated trajectory only)
python vis/vis_poses.py configs/Tanks/Ignatius.yaml
We thank Theo Costain and Michael Hobley for helpful comments and proofreading. We thank Shuai Chen and Xinghui Li for insightful discussions. Wenjing Bian is supported by China Scholarship Council (CSC).
We refer to NeRFmm, UNISURF, Vision Transformers for Dense Prediction, kitti-odom-eval and nerf-pytorch. We thank the excellent code they provide.
@inproceedings{bian2022nopenerf,
author = {Wenjing Bian and Zirui Wang and Kejie Li and Jiawang Bian and Victor Adrian Prisacariu},
title = {NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior},
journal = {CVPR},
year = {2023}
}