This work is made available under CC-BY-NC-SA-4.0 and is subject to the following statement:
Toyota Motor Europe NV/SA and its affiliated companies retain all intellectual property and proprietary rights in and to this software and related documentation. Any commercial use, reproduction, disclosure or distribution of this software and related documentation without an express license agreement from Toyota Motor Europe NV/SA is strictly prohibited.
This project uses Gaussian Splatting, which carries its original license. The GUI is inspired by INSTA. The mesh rendering operations are adapted from NVDiffRec and NVDiffRast.
You can play with a trained GaussianAvatar without downloading the dataset:
python local_viewer.py --point_path media/306/point_cloud.ply
SUBJECT=306
python train.py \
-s data/UNION10_${SUBJECT}_EMO1234EXP234589_v16_DS2-0.5x_lmkSTAR_teethV3_SMOOTH_offsetS_whiteBg_maskBelowLine \
-m output/UNION10EMOEXP_${SUBJECT}_eval_600k \
--eval --bind_to_mesh --white_background --port 60000
NOTE: During training, a complete evaluation are conducted on both the validation set (novel-view synthesis) and test set (self-reenactment) every
--interval
iterations. You can check the metrics in the commandline or Tensorboard. The metrics are computed on all images, although we only save partial images in Tensorboard.
During training, one can monitor the training progress with the remote viewer
python remote_viewer.py --port 60000
NOTE:
The remote viewer can slow down training a lot. You may want to close it or check "pause rendering" when not viewing.
The viewer could get frozen and disconnected the first time you enable "show mesh". You can try switching it on and off or simply wait for a few seconds.
After training, one can load and render the optimized 3D Gaussians with the local viewer
SUBJECT=306
ITER=300000
python local_viewer.py \
--point_path output/UNION10EMOEXP_${SUBJECT}_eval_600k/point_cloud/iteration_${ITER}/point_cloud.ply
If you find our paper or code useful in your research, please cite with the following BibTeX entry:
@article{qian2023gaussianavatars,
title={GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians},
author={Qian, Shenhan and Kirschstein, Tobias and Schoneveld, Liam and Davoli, Davide and Giebenhain, Simon and Nie{\ss}ner, Matthias},
journal={arXiv preprint arXiv:2312.02069},
year={2023}
}