search

filby89 / spectre

Official Pytorch Implementation of SPECTRE: Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos
Other
236 stars 21 forks source link

Reconstructed mesh is not stable #20

Open limei1221 opened 9 months ago

limei1221 commented 9 months ago

https://github.com/filby89/spectre/assets/21306360/d6589f90-1781-483f-a75b-3718f1fefe01

Hey I use your demo.py scripts run my video. Although the expressiveness and lip movement looks perfect the reconstructed face seems unstable, is there any way to solve this problem? Much appreciated!

filby89 commented 7 months ago

Hi, unfortunately this unstability comes from the pose/camera prediction which is taken from DECA:

https://github.com/yfeng95/DECA

In order to solve this you should check into retraining DECA to fix these pose issues or use another small network to regress on the pose/cam parameters.

limei1221 commented 7 months ago

Hi, unfortunately this unstability comes from the pose/camera prediction which is taken from DECA:

https://github.com/yfeng95/DECA

In order to solve this you should check into retraining DECA to fix these pose issues or use another small network to regress on the pose/cam parameters.

Thanks a lot for the reply! I'll try that and let you know.

marcosMeng commented 4 months ago

Have you already downloaded the spectre_model. I am unable to download this model, counld you share this model with me?

filby89 commented 4 months ago

Hey, the link for the model is:

https://drive.google.com/file/d/1vmWX6QmXGPnXTXWFgj67oHzOoOmxBh6B/view?usp=sharing