search

MRzzm / HDTF

the dataset and code for "Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset"
GNU General Public License v3.0
347 stars 65 forks source link

Is there a code to obtain 3dmm parameter dataset for training? #6

Closed kangyeolk closed 3 years ago

kangyeolk commented 3 years ago

Hi, thanks for your awesome work!!! How can I obtain 3DMM coefficients (i.e., p_mou, p_ebro...) from image/video? Or is there any reference code for this?

Thank you in advance.

MRzzm commented 3 years ago

Hi, you can obtain facial expression parameters in the following steps:

  1. Applying and downloading one 3D facial model with mean face and shape bases (BFM, LSFM etc .).
  2. Discarding their orthogonal expression bases (eigenvectors) because it is not interpretable.
  3. Sculpturing interpretable expression bases. You can follow our expression bases (we show in supplementary materials and youtube video https://youtu.be/t1QRJmpIesQ) or sculpture your own reasonable bases.
  4. For each video, regress facial expression parameters with eq.(2) in the paper.

You can employ one technical artist to sculpture expression bases and refer to this repo to reconstruct 3D face.