search

Coyote-A / ultimate-upscale-for-automatic1111

GNU General Public License v3.0
1.65k stars 163 forks source link

Add checkboxes to "Use Seed From Image" and "Use Prompt from image" for ideal batch processing #73

Open marcsyp opened 1 year ago

marcsyp commented 1 year ago

First of all, thanks for this extension -- it's amazing!!

One of the things that would really make this extension complete involves upgrades that would allow for effective batch processing, which is currently extremely limited.

Don't know how complicated this is on your end, but it would be really great for batch processing to be able to override the seed in the webui to be used for upscaling by pulling directly from the image metadata. This would allow each image to get the seed that was used originally to generate the image, regardless of the order in which is provided (for instance, a folder with 10 selected images out of a run of 200).

Currently, the batch processing will use the same seed for every image, which works great for one image but the rest will have some deviance (unwanted) from the original generation, polluting the upscaling results.

Additionally, I often use dynamic prompts for my image generation, so if the seed could be used to pull the correct prompt for that seed from what is in the webui, that would be ideal (so that you can make modified versions of the dynamic prompt for upscaling if desired) -- though I suspect that this is fairly complicated from a code standpoint since it involves interaction with another extension. As a fallback, a checkbox that would pull the prompt that was used for the image from the metadata.

Together, these two upgrades would allow you to execute full batch runs with dynamic prompts and arbitrary seeds with a perfect match for upscaling purposes.

Thanks for your consideration!

marcsyp commented 1 year ago

Good news!

I poked around the extension code and the stable diffusion source code and managed to put together a few ideas that I ran past ChatGPT. We collaborated and I have successfully implemented all of the above in a custom forked version of the SDSU script.

It turns out that the Dynamic Keywords script is not stable with respect to seed, so even though I got it working with dynamic prompts, it was necessary to implement a checkbox system for choosing whether to pull Seed, Prompt, and/or Negative Prompt from the image, so you can mix and match those as desired, and those are implemented in the UI. I am happy to contribute this code back to this project as my first contribution to the A1111 ecosystem, just let me know and I can pass it along. I only modified the extension's script, but I have not cloned the project so I can't do a PR at the moment. They are pretty simple changes though.

I am currently running a batch USDU on 200 images with different prompts and seeds, and it is performing beautifully!

Cheers, Marc

jau123 commented 1 year ago

@marcsyp Hi there,

I came across your message and noticed that you've successfully implemented a batch processing feature in the project. That's an impressive accomplishment! I was wondering if you could kindly share some insights on how you achieved this or provide some guidance to help me understand the process better. Your input would be greatly appreciated.

Thank you in advance,

andupotorac commented 1 year ago

While we're at it, it would also be cool to allow us to use the same image with ControlNet's Tile, and specific settings, for the same reasons - to use batch processing.

andupotorac commented 1 year ago

Marc, were you able to submit this to the core project? I'd love to give it a run, but curious if it would be possible to pass the image to Controlnet with Tile (and some specific settings) too. I found it helps to avoid Seams when used this way.