Closed whosawhatsis closed 1 year ago
I also find it a little frustrating when I decide to cut a batch generation short and therefore kill it before GFPGAN/ESRGAN hits any of the gens.
We implemented GFPGAN and ESRGAN very early. During the initial implementation, the post processing effects ran right after image generation. But the issue with this was that they obviously took more memory and as a result the performance of the image generation was itself taking a huge hit .. and in a lot of cases just outright crashing due to memory issues -- using 8GB cards as the standard because that's what a large chunk of people have.
And given most people generate 1 image at a time, we decided the work around here was to run them AFTER the image generation so we can avoid the crashes. This obviously meant that multiple image generations would have all the images processed after the completion of the image generation because the image generation loop wouldn't end before that.
But obviously a LOT has changed since then. The memory optimization work that has been done has greatly lowered memory footprint of the image generation process. Maybe it is now a viable strategy once again to look at post processing effects running right after image generation.
I'll take a look at this over the weekend and see if there's a way to implement this without a switch. But if not, I guess we can still add a switch for users who have systems that are capable of handling this.
I'll note that in the web UI, the sequence is as you request - generation1, upscale1, facefix1, generation2, upscale2, facefix2, and so on.
If you cancel before an image has gone through request postprocessing, though, it is not saved. I suppose it should save it if you cancel after generation but before post?
Are you sure? I just got the webui running and tried it. I set it to generate 10 images. It did that, then began upscaling them. Maybe you're running an older version or a different branch? I just installed mine two days ago...
I suppose you are on the main
branch? On development
, there is a brand new UI which works as I described (that's how it should work, anyways).
Ah, cool, I'll check that out.
Yes, the development version of the webui does do the upscaling for each image, which I like a lot better. I still think it would be useful to be able to do that in parallel with generating the next image, though, since (at least on apple silicon), one uses the CPU and the other uses the GPU, so they don't step on each other much.
@Kyle0654 With the new backend, will the workers process jobs in parallel e.g. iteration 1's postprocessing can happen in parallel with interation 2's generation?
@Kyle0654 With the new backend, will the workers process jobs in parallel e.g. iteration 1's postprocessing can happen in parallel with interation 2's generation?
It could be done, but it's a challenging problem with the new node-based stuff, since I can't readily tell what resources would need to be locked to process a node, and those could change based on settings.
When generating large sets of images, my GPU is pegged while generating images (txt2img), but my CPU is idle. When that finishes, it starts running Real-ESRGAN and GFPGAN on all of those images, which uses close to half of my CPU time and none of my GPU for nearly as long.
This is on an M1 Max, and the optimum might be different when using CUDA, but it would be nice to have an option to run the upscaling for one image in parallel with generating the next image, rather than doing all of the generation, then all of the upscaling after that finishes.
A related issue is that, if I decide to stop the image generation early, nothing gets upscaled. This would also be fixed by my suggestion above. Even if it instead alternated, generating one image, then upscaling it, then generating the next image, and so on, ending generation early could still result in upscaled images. This would result in alternating between CPU and GPU tasks, which might if nothing else, at least have the benefit of reducing the maximum temperature (and thus fan noise level) of the computer during the whole process.