search

kabachuha / sd-webui-text2video

Auto1111 extension implementing text2video diffusion models (like ModelScope or VideoCrafter) using only Auto1111 webui dependencies
Other
1.28k stars 106 forks source link

[Feature Request]: Add upscaling, frame interpolation from Deforum + Face restore #14

Open kabachuha opened 1 year ago

kabachuha commented 1 year ago

Is there an existing issue for this?

What would your feature do ?

Since the resulting videos are far smaller than the usual StableDiffusion generated ones (and sometimes flickery as well), they will benefit greatly from adding upscaling and frame interpolation, just like it's done in vanilla Deforum

https://github.com/deforum-art/deforum-for-automatic1111-webui/blob/automatic1111-webui/scripts/deforum_helpers/upscaling.py

https://github.com/deforum-art/deforum-for-automatic1111-webui/blob/automatic1111-webui/scripts/deforum_helpers/frame_interpolation.py

Proposed workflow

  1. Go to text2video tab
  2. Generate animation
  3. Have it upscaled/interpolated automatically or manually in the output tab

Additional information

No response

volotat commented 1 year ago

I would also suggest to use FTVSR algorithm for video upscaling. As it is specifically built to upscale low quality videos taking into account compression artifacts. It should also work much faster than upscaling video frame by frame.