search

Kahsolt / stable-diffusion-webui-prompt-travel

Travel between prompts in the latent space to make pseudo-animation, extension script for AUTOMATIC1111/stable-diffusion-webui.
The Unlicense
248 stars 22 forks source link

Feature Request: Support Upscaling before generating video. #12

Closed Hellisotherpeople closed 1 year ago

Hellisotherpeople commented 1 year ago

Right now, for both this extension and for seed travel, a user who wants to upscale their animation will need to individual (or in a batch) upscale each image and then will have to convert these images into an mp4 or .gif themselves.

Maybe we should support allowing a user to do this in the UI before the final generated video. Besides, you'll be able to show off 8K animations really easily to motivate folks to use it!

Kahsolt commented 1 year ago

Good advice, will be implemented soon~

Kahsolt commented 1 year ago

Well, this is done. However, ui components might not be that tunable but enough for basic cases ;)

Hellisotherpeople commented 1 year ago

Exciting! Thank you for the quick response! I'm super excited to see this feature, and I'll even make a post on my Linkedin (which has a decent amount of followers) talking about this once it's merged!!!!

Kahsolt commented 1 year ago

Kinda shocked, but really appreciate your attention to this extension~ 😸 To make coding simple, current upscaling is implemented in a post-processing manner. Hence the generated images are still in low-resolution, but exported video will be high-resolution. Yet another way to get a high resolution stuff is that, you could try using this extension together with the hires.fix feature, though it would be much slower.

Hellisotherpeople commented 1 year ago

One small request, can we set the scaling multiplier as high as 8 (or higher if it can go even further)? The regular upscaler supports going that high. I was hoping to do 4K videos Otherwise this is awesome and works perfectly!

Kahsolt commented 1 year ago

Get it, let me check the default upper limit setting in webui.

Kahsolt commented 1 year ago

Now in the webui script panel, the upscaling multiplier can reach 16, values higher than x4 will be auto broken down like x8 -> x4 x2, x16 -> x4 x4, x12 -> x4 x3 (the first pass is always x4).
If you still could not set values higher than 4 in the ui widgets, check option "customscript/prompt_travel.py/txt2img/Upscale ratio/maximum" in local setting file ui-config.json ;)

⚠ x16 will probably make nothing but a all black video if your upsacled image resolution is more than 4096 :( ℹ I tested the simple Lanczos, it's works but bad quality; when I want to test the bundled ESRGAN-4x, my webui just stucked and seemingly get into somewhere a deadloop 🤔

Another road, I encourage people to try the new standalone post-porcessing pipeline. To get a x16 video, you just need to: