search

Hangz-nju-cuhk / Talking-Face_PC-AVS

Code for Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation (CVPR 2021)
Creative Commons Attribution 4.0 International
922 stars 168 forks source link

result does not look like src #3

Open molo32 opened 3 years ago

molo32 commented 3 years ago

source has 700 frames aligned so:

el csv : /content/Talking-Face_PC-AVS/misc/Input/faf/ 700 /content/Talking-Face_PC-AVS/misc/Pose_Source/517600078 160 /content/Talking-Face_PC-AVS/misc/Audio_Source/00015.mp3 /content/Talking-Face_PC-AVS/misc/Mouth_Source/681600002 363 dummy"

how to improve the result?

https://user-images.githubusercontent.com/55426197/115946954-f4911600-a49a-11eb-9d71-7a5b66145d90.mp4

Hangz-nju-cuhk commented 3 years ago

Hi, the problem might be caused partially by the alignment of the reference input and partially by the exaggerated initial expression. The model might find it difficult to locate the eyes given its leaning angles. It would be better if the single input image can be aligned by the key points.

Kasumigaoka-Utaha commented 3 years ago

The problem becomes severe if we choose some faces with some strange angles

https://user-images.githubusercontent.com/45712215/118389744-e8454680-b65d-11eb-93f0-138f25265e59.mp4

Hangz-nju-cuhk commented 3 years ago

@Kasumigaoka-Utaha why not read our README first and figure out how to align your own image??

Kasumigaoka-Utaha commented 3 years ago

Thx, but the results also seems strange, not like what you have shown in the demo

https://user-images.githubusercontent.com/45712215/118396241-37e93980-b681-11eb-84df-dc22f64e3324.mp4