search

Weizhi-Zhong / IP_LAP

CVPR2023 talking face implementation for Identity-Preserving Talking Face Generation With Landmark and Appearance Priors
Apache License 2.0
679 stars 76 forks source link

Lip is so small and can not handle long range face #29

Open yuanjunchai opened 1 year ago

yuanjunchai commented 1 year ago

Hi Dear, Your work is so excellent! But after test many videos, I find generated lip is so small and it can not handle long range face. Could you have any ideas to solve these issues?

KingStorm commented 1 year ago

I guess the small lip is related to head motion (i.e., your head motion is nearly still)

what do you mean about the long range face?

yuanjunchai commented 1 year ago

Hi Jianhao!

First, I do not understand the relationship between head motion and lip openness. You mean the more head motion, the better lip open size? Second, the long-range is compared to the low-range face, which means the small face, captured by long range lens, appears in a high-resolution video.

Jianhao Ye @.***> 于2023年8月5日周六 22:44写道：

I guess the small lip is related to head motion (i.e., your head motion is nearly still)

what do you mean about the long range face?

— Reply to this email directly, view it on GitHub https://github.com/Weizhi-Zhong/IP_LAP/issues/29#issuecomment-1666524414, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGC5JRSFB4P4ZZ4WY4PHGSLXTZL33ANCNFSM6AAAAAA2565Y4A . You are receiving this because you authored the thread.Message ID: @.***>

KingStorm commented 1 year ago

Yeah I was to mean there are entanglement between head motion and lip syncing, It depends on your training dataset. If in your training dataset, the speaker only speaks when head moves. This would make the model to learn that relationship, i.e., speak when head motion moves -> lip is small when head motion is still.

BTW, what do you observer in your long-range face case?

sunjian2015 commented 8 months ago

lip is small when head motion is still

I have encountered this problem, how should i solve it?