ACM Transactions on Graphics (Proceedings of SIGGRAPH), July 2023.
Baptiste Nicolet
·
Fabrice Rousselle
·
Jan Novák
·
Alexander Keller
·
Wenzel Jakob
·
Thomas Müller
This repository contains code examples to reproduce the results from the article:
Baptiste Nicolet and Fabrice Rousselle and Jan Novák and Alexander Keller and Wenzel Jakob and Thomas Müller, 2023. Recursive Control Variates for Inverse Rendering. In Transactions on Graphics (Proceedings of SIGGRAPH) 42(4).
It uses the Mitsuba 3 differentiable renderer.
This code is released under the BSD 3-Clause License. Additionally, if you are using this code in academic research, please cite our paper using the following BibTeX entry:
@article{Nicolet2023Recursive,
author = {Baptiste Nicolet and Fabrice Rousselle and Jan Novák and Alexander Keller and Wenzel Jakob and Thomas Müller},
title = {Recursive Control Variates for Inverse Rendering},
journal = {Transactions on Graphics (Proceedings of SIGGRAPH)},
volume = {42},
number = {4},
year = {2023},
month = aug,
doi = {10.1145/3592139}
}
This code was tested on Ubuntu 22.04 with an NVIDIA RTX 4090 GPU. NVIDIA driver version 525.60.13 was used with CUDA 12.0.
Mitsuba 3 was compiled with Clang++ 11.0.0 and the provided scripts were run with Python 3.9.12.
The cuda_ad_rgb
Mitsuba variant was selected, although the llvm_ad_rgb
variant is also compatible in principle.
This implementation relies on modifications to the Mitsuba source code, which are available on the unbiased-volume-opt
branch of the mitsuba3
repository.
Please make sure to checkout the correct branch as follows.
Note the --recursive
and --branch
flags:
# Cloning Mitsuba 3 and this repository
git clone --recursive https://github.com/mitsuba-renderer/mitsuba3 --branch unbiased-volume-opt
git clone --recursive https://github.com/rgl-epfl/recursive_control_variates
# Building Mitsuba 3, including the project-specific modifications
cd mitsuba3
mkdir build && cd build
cmake -GNinja ..
ninja
The cuda_ad_rgb
and llvm_ad_rgb
variants should be included by default.
Please see the Mitsuba 3 documentation for complete instructions on building and using the system.
The scene data must be downloaded and unzipped at the root of the project folder:
cd recursive_control_variates
wget https://rgl.s3.eu-central-1.amazonaws.com/media/papers/Nicolet2023Recursive.zip
unzip Nicolet2023Recursive.zip
rm Nicolet2023Recursive.zip
ls scenes
# The available scenes should now be listed (one directory per scene)
Navigate to this project's directory and make sure that the Mitsuba 3 libraries built in the previous step are made available in your current session using setpath.sh
:
cd recursive_control_variates
source ../mitsuba3/build/setpath.sh
# The following should execute without error and without output
# (use the variant 'llvm_ad_rgb' if your system does not support the CUDA backend):
python3 -c "import mitsuba as mi; mi.set_variant('cuda_ad_rgb')"
From here, the script python/run_experiment.py
can be used to run inverse rendering examples using different methods:
python3 run_experiment.py teapot --method cv_ps
The script expects a scene name (teapot
in the example above). The scene
configurations are defined in experiments.py
. You can also specify with which method you want to run the optimisation:
baseline
: Optimisation with standard differentiable rendering integrators.cv_pss
: Optimisation using our control variates, with the 'primary sample space' implementation (see the paper for an explanation).cv_ps
: Optimisation using our control variates, with the 'path space' implementation.If no method is specified, both baseline
and cv_ps
optimisations will be run.
We provide scripts to reproduce the figures from the paper. These are located in
the figures
subfolder. Each figure has its own subfolder, with a
generate_data.py
, that will run the relevant experiments needed to generate
the figure. Then, one can run the figure.ipynb
notebook to generate the figure.
In tutorial.ipynb
, we show how our control variates can be easily integrated
into an optimisation. One merely needs to use the TwoState
adapter BSDF (resp.
medium) for the surface (resp. medium) being optimised, and use the twostate
variant of the prb
(resp. prbvolpath
) integrator.
This README template was created by Miguel Crespo, and its structure inspired by Merlin Nimier-David. Many figure generating utility functions were written by Delio Vicini.
The volpathsimple
integrator was implemented by Merlin Nimier-David
Volumes, environment maps and 3D models were generously provided by JangaFX, PolyHaven, Benedikt Bitterli and vajrablue.