search

semchan / UPST-NeRF

code for UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene
https://semchan.github.io/UPST_NeRF/
68 stars 7 forks source link
3d-scene neural-radiance-fields photorealistic-style-transfer

readme

UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene

UPST-NeRF(see our paper and project page )is capable of rendering photorealistic stylized novel views with a consistent appearance at various view angles in 3D space.

Qualitative comparisons

Installation

git clone https://github.com/semchan/UPST-NeRF.git
cd UPST-NeRF
pip install -r requirements.txt

Pytorch and torch_scatter installation is machine dependent, please install the correct version for your machine.

Dependencies (click to expand) - `PyTorch`, `numpy`, `torch_scatter`: main computation. - `scipy`, `lpips`: SSIM and LPIPS evaluation. - `tqdm`: progress bar. - `mmcv`: config system. - `opencv-python`: image processing. - `imageio`, `imageio-ffmpeg`: images and videos I/O.

Download: datasets, trained models, and rendered test views

Directory structure for the datasets (click to expand; only list used files) data ├── coco # Link: http://cocodataset.org/#download │ └── [mscoco2017] │ ├── [train] │ └── r_*.png ├── nerf_synthetic # Link: https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1 │ └── [chair|drums|ficus|hotdog|lego|materials|mic|ship] │ ├── [train|val|test] │ │ └── r_*.png │ └── transforms_[train|val|test].json │ │ └── nerf_llff_data # Link: https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1 └── [fern|flower|fortress|horns|leaves|orchids|room|trex]

Synthetic-NeRF datasets

We use the datasets organized by NeRF. Download links:

LLFF dataset

We use the LLFF dataset organized by NeRF. Download link: nerf_llff_data.

Train

To train fern scene and evaluate testset PSNR at the end of training, run:

$ python run_upst.py  --config configs/llff/fern.py  --style_img ./style_images/your_image_name.jpg

Evaluation

To only evaluate the trained fern, run:

$ python run_upst.py --config configs/llff/fern.py --style_img ./style_images/your_image_name.jpg --render_style --render_only --render_test --render_video

We also share some checkpoints for the 3D senes on llff dataset in baidu disk. You can download and put it into "./logs" for evaluation.

link：https://pan.baidu.com/s/18z70qCdRXjm7j1EyCh63Gw

code：1234

Acknowledgement

Thanks very much for the excellent work of DirectVoxGO, our code base is origined from an awesome DirectVoxGO implementation.