kabachuha / sd-webui-text2video

Auto1111 extension implementing text2video diffusion models (like ModelScope or VideoCrafter) using only Auto1111 webui dependencies
Other
1.28k stars 108 forks source link

[Feature Request]: Support SD Video (SVD-xt) #237

Open nickmitchko opened 8 months ago

nickmitchko commented 8 months ago

Is there an existing issue for this?

What would your feature do ?

I would like to request a feature that supports the new Stable Video Diffusion model, which can be found at this link: https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt. This new model represents a significant advancement in generative video technology and has the potential to greatly enhance the user experience.

The feature extension should include the following capabilities:

  1. Ability to specify the number of frames: Users should be able to choose the desired number of frames in the generated video, allowing them to control the length and detail of the output.
  2. Adjustable resolution: Provide the option to select the resolution of the generated video, giving users more control over the quality and file size of the output.
  3. Upload pane for source image: Incorporate an upload pane for a source image, which can be used as a reference for the video model. This will enable users to easily input their desired image and create a video based on it.
  4. . Integration with img2txt tab: Add a button that allows users to seamlessly pull images from the img2txt tab within the application, streamlining the process of creating a video from an existing image.

For reference, the sample code can be found at this link: https://github.com/Stability-AI/generative-models. I have lots of VRAM, would be happy to test.

Thank you for considering this feature request, and I look forward to the continued development and improvement of the sd-webui-text2video extension.

Proposed workflow

  1. Press [txt2video]
  2. Add Model Type [SVD-xt]
  3. Inherit the same settings as the txt2vid sub tab.

Additional information

No response