search

Zejun-Yang / AniPortrait

AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation
Apache License 2.0
4.25k stars 525 forks source link

Audio2video any hints on lip sync #59

Open fralik opened 3 months ago

fralik commented 3 months ago

I am using audio2video script with a song, i.e. audio contains music and voice. The produced video doesn't have any lip movement. On the face mesh I can see small lip movement, but not on the image. Any hints on improving this?

Could it be because of default pose_temp?

Zejun-Yang commented 3 months ago

pose_temp only provides head pose. And large amplitude mouth movements would result in improved lip synchronization in the generated video.

Mrkomiljon commented 1 month ago

@Zejun-Yang how about size ? what kind of img size i have to use audio2video script ? I need clear lip sync and at least shoulder movement. please give me some advices.

Zejun-Yang commented 1 month ago

@Zejun-Yang how about size ? what kind of img size i have to use audio2video script ? I need clear lip sync and at least shoulder movement. please give me some advices.

The code will resize the input image to 512*512 if you set default -W and -H. So you can use any square portrait images, and make sure the face is large enough in the image.