Face Viewer

Paper | Project Page

Real-time Face Viewer for the paper "3D Gaussian Blendshapes for Head Avatar Animation". The training code can be found here

Build

We have tested building on Ubuntu 18.04 with CUDA 11.8.
Requirements:

• An NVIDIA GPU
• gcc/g++ 11.0 or higher
• CMake v3.19 or higher

Install dependencies

sudo apt-get install zlib1g-dev libfreeimage3 libfreeimage-dev libglew-dev libglu1-mesa-dev libglfw3-dev

Then, use CMake to build the project

mkdir build
cd build
cmake ..
make
cd ..

Run viewer by

./build/bin/FaceViewer_FLAME --dataset=DATA_PATH/params --model=MODEL_PATH

The building on Windows is also possible (VS2019 + CUDA 11.8) but may require some modification to CMakeLists. Alternatively, We provide a pre-built binary here.

Data process

You need to create an account on the FLAME website and download FLAME 2020 model. Please unzip FLAME2020.zip and put generic_model.pkl under ./data/FLAME2020.

Then run

python scripts/extract_tensor.py -i ./data/FLAME2020/generic_model.pkl -o ./data/FLAME2020

Run script to transform dataset format for viewer.

python scripts/extract_traj.py -i DATA_PATH/checkpoint -o DATA_PATH/params  

The datasets can be found here. The pre-trained models are available here.

How to use

Basic Control
Use left mouse button to rotate, right mouse button to pan, and middle mouse mouse button to zoom in and out.

Set Current Frame
Select current frame under panel Render Setting->Frames.
You can also use arrow keys to forward and rewind.
Key P to start/stop playing animation.

Rendering Mode
Use the number keys to select the rendering mode, 3 for Gaussian head avatar, and 1 for corresponding FLAME mesh.

Others
Key C to enable/disable rendering coordinate.
Key T to enable/disable rendering camera trajectory.

Important hints

To reproduce the 370fps performance reported in the paper, please comment out ENABLE_VSYNC at line 36 and uncomment line 1810 in FaceViewer_FLAME.cpp and rebuild. By default, we have optimized out some unnecessary computations and enabled vsync.

If your goal is to have a side-to-side comparison with GT, you may comment out lines 1114-1116 in FaceViewer_FLAME.cpp for correct alignment and rebuild. You may also adjust window size, for example, set --winWidth=512 and --winHeight=512.