search

continue-revolution / sd-webui-animatediff

AnimateDiff for AUTOMATIC1111 Stable Diffusion WebUI
Other
3.09k stars 257 forks source link

[Feature]: Break up Hires images into smaller batches during hires-fix #102

Open CatJak opened 1 year ago

CatJak commented 1 year ago

Expected behavior

Current behavior: All images in the Hires pass are done at the same time often leading to OOM errors due to the sheer number of images Would it be possible to set a option where the user can set the batch size for the Hires pass?

continue-revolution commented 1 year ago

I‘m not sure how "hires-fix" works. So expect to wait for several weeks before this is available.

zappityzap commented 1 year ago

A workaround outside of this extension could be running the individual frames through an img2img batch, then combine with ffmpeg.

GumGum10 commented 1 year ago

You could try controlnet tile batch input to upscale too

rkfg commented 1 year ago

I implemented hires fix for this extension before in a local fork. See here, it's outdated already because most features from that fork were implemented in upstream so I stopped developing it. HRF is surprisingly harder than it sounds because we need to reinit many things if we change the batch size. In my case I simply upscale one by one, the GPU is utilized fully so I don't think we'd win much time doing bigger batches. Basically everything needs to be reinitialized. While I got it to work there were issues with consistency: either you have to use a low denoise strength (0.2 and below) or some details start flickering because every frame is processed independently so the new details from the added noise can be different from frame to frame. And adding the motion module to the mix is beyond my capabilities, sorry 😅

However, I noticed that after some optimizations in A1111 (I assume) it's now possible to generate 16 images with HRF in one go! It takes 20 Gb of VRAM but doesn't OOM if you have 24. Before even 5 images spiked at some point and caused OOM because 4 images worked and only required about 6-8 Gb but the spike was up to 15-17 Gb.

Currently using HRF with AnimateDiff causes it to crash with a cryptic CUDA error, at least in my experience.