jbilcke-hf / clapper

Clapper.app, a video synthesizer and sequencer designed for the age of AI cinema
https://clapper.app
GNU General Public License v3.0
2.08k stars 194 forks source link

Add support for EchoMimic for lip-syncing with realistic 3D head animations #27

Closed jbilcke-hf closed 4 months ago

jbilcke-hf commented 4 months ago

Ticket specification to be completed..

https://x.com/fffiloni/status/1815424286460354753

(note: we already have some code to call Gradio spaces)

EDIT: humm, I have tested it but it seems to be very slow, and only work on square images. And conventional face poses.

So, I don't know!

lalalune commented 4 months ago

This is trivial in ComfyUI, IMO a more robust ComfyUI integration will get these workflows almost for free

jbilcke-hf commented 4 months ago

💯 true, indeed ComfyUI integration can be the answer to a lot of those "how to generate X or Y" questions

jbilcke-hf commented 4 months ago

marking as duplicate of https://github.com/jbilcke-hf/clapper/issues/5