search

invoke-ai / InvokeAI

Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
23.38k stars 2.4k forks source link

[enhancement]: Add extension support, I want to install AnimateDiff in InVoke. #7080

Open oaycreations opened 5 days ago

oaycreations commented 5 days ago

Is there an existing issue for this?

What should this feature add?

I would like you to add extension support from the original Gradio Interface. I really need the AnimateDiff extension, but it stopped working for me in the original Gradio.

Alternatives

Motion models for this extension can be downloaded from InVoke from the original HuggingFace repository, but when the download is complete, an error is returned:

[2024-10-09 12:25:07,659]::[ModelInstallService]::ERROR --> Model install error: https://huggingface.co/guoyww/animatediff/resolve/main/mm_sd_v15.ckpt InvalidModelConfigException: Unable to determine model type for /home/oaycreations/invokeai/models/tmpinstall_ndwdux_2/mm_sd_v15.ckpt

2024-10-09_12-50

And not only me one: https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/434 https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/330 https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/1307

Additional Content

I left these motion models loading overnight yesterday, but today I saw that only 1 of them loaded. Apparently, without the extension, they are simply not recognized by InVoke.

I think it would be useful to add an additional tab with extensions in the upper left corner, so that adding them is easy and clear. However, this will require the implementation of Gradio code in your Web Interface code.

hipsterusername commented 5 days ago

Can you clarify what you mean by the original gradio interface? Invoke has never supported Animated-diff.

Beyond that, custom nodes are supported, and anyone is welcome to tackle the project of adding it -- We'd support any necessary updates to handling media beyond images.

psychedelicious commented 5 days ago

Invoke has a custom UI build in React. It is not built with Gradio, and there is no interoperability with Gradio for the UI specifically. The backend is python and can therefore use any python libraries. It's certainly possible to implement AnimateDiff, especially if there is a python library or reference implementation. However, we'd have some work to do in the UI to support video stuff.