search

Inferencer / LipSick

🤢 LipSick: Fast, High Quality, Low Resource Lipsync Tool 🤮
The Unlicense
132 stars 17 forks source link

Will it work for image (single frame video)? #34

Open tonyabracadabra opened 4 weeks ago

tonyabracadabra commented 4 weeks ago

Thanks for the amazing project! Just wondering if it's possible to create a full video with audio-driven lip sync from a single frame of an image like what https://github.com/fudan-generative-vision/hallo does?

Also I am curious on the benchmark (primarily speed and memory usage, as I am really satisfied with the quality from the demo!) comparing LipSick to other models like wav2lip and video retalking.

Inferencer commented 4 weeks ago

you could create a vid out of a single frame but there would be only lip movement not head movement which will look weird, i recommend lipsyncing on a good source vid then using it to drive an image with https://github.com/KwaiVGI/LivePortrait

tonyabracadabra commented 4 weeks ago

you could create a vid out of a single frame but there would be only lip movement not head movement which will look weird, i recommend lipsyncing on a good source vid then using it to drive an image with https://github.com/KwaiVGI/LivePortrait

Yea that sounds like a plan, will live portrait be integrated into this project?