AIFSH / ComfyUI-MuseTalk_FSH

the comfyui custom node of MuseTalk to make audio driven videos!
Other
14 stars 5 forks source link

High-quality zero-shot lipsync pipeline built on MuseTalk #11

Open mvoodarla opened 1 month ago

mvoodarla commented 1 month ago

Hey folks! My team has been exploring zero-shot lipsyncing for a bit and we think we've improved on MuseTalk's quality quite a bit by using a pipeline built with CodeFormer and LivePortrait as well. Here's an example.

https://github.com/user-attachments/assets/cfabcd9f-92e0-4c52-b786-77fc63eef81b

We wrote a technical blog on it: https://www.sievedata.com/blog/sievesync-zero-shot-lipsync-api-developers

Hope to put out an OSS repo soon too :)

Anything we don't talk about in the blog that we should in our repo release?