search

kabachuha / sd-webui-text2video

Auto1111 extension implementing text2video diffusion models (like ModelScope or VideoCrafter) using only Auto1111 webui dependencies
Other
1.28k stars 106 forks source link

[Feature Request]: Blending in Stable Diffusion #149

Open neverix opened 1 year ago

neverix commented 1 year ago

Is there an existing issue for this?

What would your feature do ?

Modelscope and SD use the same autoencoder. miniSD is a version of SD that has the same resolution as Modelscope. It's possible to blend scores from the two models together to make the individual frames look better.

Proposed workflow

I'm open to suggestions, here's an example workflow:

  1. Click a checkbox to enable SD blending
  2. Set weights for video and image model
  3. ???
  4. Get video output

Additional information

No response

kabachuha commented 1 year ago

Yes, it kinda is in the plans to make t2v/Deforum integration. We are going to make a some sort of API for the latter to allow other projects interference into its workflow at various generation stages and then it will be much easier