Support SOTA Difussion Based talking head repos like Echomimic if not already supported

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.

https://huggingface.co/docs/diffusers

Apache License 2.0

25.98k stars 5.35k forks source link

Support SOTA Difussion Based talking head repos like Echomimic if not already supported #9783

Open souvikg544 opened 4 days ago

souvikg544 commented 4 days ago

I have seen that Hugging Face doesnt have much of the Talking Head Repositories under its hood. Would love to contribute and integrate it within the diffusers pipeline. Link to the original github repo - https://github.com/BadToBest/EchoMimic .

a-r-r-o-w commented 3 days ago

Thanks for the suggestion! Currently, there are no plans to support models apart from the standard text/image/control/video - to - image/video/audio/3D. I also notice that they already provide Diffusers-like pipelines to make usage convenient.

We might eventually consider supporting these kinds of models more readily as add-on diffusers packages once we have a more stable Modular Diffusers (see PR 9672).