pcl3dv/PSAvatar - Githubissues

# PSAvatar: A Point-based Shape Model for Real-Time Head Avatar Animation with 3D Gaussian Splatting Zhongyuan Zhao^1,2 Zhenyu Bao^1,2, Qing Li¹, Guoping Qiu^3,4, Kanglin Liu^1,* ¹Pengcheng Laboratory ²Peking University ³University of Nottingham ⁴Shenzhen University ^*corresponding author: max.liu.426@gmail.com ### [Paper](https://arxiv.org/abs/2401.12900) | Project | [Video](https://github.com/user-attachments/assets/1d075705-11ac-43ee-a771-16f2d1621083) | Code ( is coming soon... )

*Abstract: Despite much progress, achieving real-time high-fidelity head avatar animation is still difficult and existing methods have to trade-off between speed and quality. 3DMM based methods often fail to model non-facial structures such as eyeglasses and hairstyles, while neural implicit models suffer from deformation inflexibility and rendering inefficiency. Although 3D Gaussian has been demonstrated to possess promising capability for geometry representation and radiance field reconstruction, applying 3D Gaussian in head avatar creation remains a major challenge since it is difficult for 3D Gaussian to model the head shape variations caused by changing poses and expressions. In this paper, we introduce PSAvatar, a novel framework for animatable head avatar creation that utilizes discrete geometric primitive to create a Point-based Shape Model (PSM) and employs 3D Gaussian for fine detail representation and high fidelity rendering. The PSM uses points instead of meshes for 3D representation to achieve enhanced representation flexibility. Specifically, PSM first converts the FLAME mesh to points by sampling on the surfaces as well as off the meshes to enable the reconstruction of not only surface-like structures but also complex geometries such as eyeglasses and hairstyles. By aligning these points with the head shape in an analysis-by-synthesis manner, the PSM makes it possible to utilize 3D Gaussian for fine detail representation and appearance modeling, thus enabling the creation of high-fidelity avatars. We show that PSAvatar can reconstruct high-fidelity head avatars of a variety of subjects and the avatars can be animated in real-time ($\ge$ 25 fps at a resolution of 512 $\times$ 512 ).