gobig changes seed and prompt when creating gobig version

[rockerBOO]

Big fan of this repo and with the new memory changes added I was able to create 1024x1024 images with the image creation. Thanks for adding this, as the many features of this repo have been very fun to play with.

I was starting to play with the gobig thing with the realesrgan option. But when making the big image, it creates odd images. It seems to re-evaluate the prompt and also increment the seed. This doesn't seem like what it should be doing.

My expectation is when making a gobig image, it should freeze the seed and the prompt. Also, it only seems to make a gobig version for the last image in a batch/iter, but would be cool if it did it for each one.

No worries if this is not ideal or otherwise not desired to do.

{
  "prompt": "A cute ghost seen lost, deep in the woods at night, _lighting_, _adjective_, _detail_, _genre_, _style_, in _environment_, _colors_, by _artist_,  _artist_, and _artist_",
  "batch_name": "ghost-2-gobig", // Directory to save to inside out_path
  "out_path": "./out", // base output directory, default is "out" inside your prs folder.
  "width": 1024, // resolution (not used when doing init_image)
  "height": 1024, // resolution (not used when doing init_image)
  "steps": 20, // Number of steps to do
  "scale": 7.5, // Affects how strongly the prompt is applied, a value from 6.0 to 9.0 is sensible.
  "seed": "random", // Can be set to a specific number (no quotes), is incremented with each batch
  "frozen_seed": false, // When true, the seed will be kept the same for every image (useful when randomizing prompts)
  "n_batches": 2, // Number of overall batches of images to make in this run (seed changes every batch)
  "n_iter": 2, // number of images to make within a batch (seed remains the same for each iteration)
  "variance": 0.6, // How much to vary each image iteration within a batch (only applies if n_iter is 2 or more)
  // Init image stuff
  "init_image": null, // the file to use for a starting point (leave as null for no init)
  "init_strength": 0.4, // how strongly the init should be applied (essentially skip_steps)
  // Go BIG stuff
  "gobig": true, // A true setting will trigger the gobig upscaling process (or, use --gobig on the command line)
  "gobig_maximize": false, // auto-extend go_big into otherwise wasted space
  "gobig_overlap": 64, // number of pixels to blend against other areas in go_big
  "gobig_realesrgan": true, // optionally use RealESRGAN to handle initial resize, does not work on the docker container currently
  "gobig_keep_slices": false, // optionally keep the individual slice images
  "esrgan_model": "realesrgan-x4plus", // the model to use for esrgan upscaling
  "esrgan_exe": "./real/realesrgan-ncnn-vulkan", // the location of the realesgarn executable 
  // Generally, you can ignore most of the rest of this
  "method": "k_dpm_2_ancestral", // sampling method to use. Options: "k_lms", "k_dpm_2_ancestral", "k_dpm_2", "k_heun", "k_euler_ancestral", "k_euler", "ddim"
  "eta": 0.0, // currently does not seem to do anything
  "from_file": null, // load prompts from this file instead of using a single prompt. Each prompt is rendered n_batches times
  "cool_down": 0.0, // number of seconds to pause between images, to keep GPU cool
  "checkpoint": "./models/sd-v1-4.ckpt",
  "use_jpg": false, // set this to save your output in JPG format to save space
  "hide_metadata": false, // set to true if you don"t want your prompt and seed in the PNG output metadata
  "settings_dir": "./settings", // set to the directory of your settings for prompt randomizer
  "save_settings": true // set to true if you want a settings file like this saved with every image
}

Partial output

...
Starting render!
Global seed set to 9422631
Sampling:   0%|
Prompt for this image:
   ['A cute ghost seen lost, deep in the woods at night, subsurface scattering, functional, sharp focus, historic, hyperrealism
t, by John Atkinson Grimshaw,  April Gornik, and Sonia Delaunay']

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12

Output saved as "./out/ghost-2-gobig/ghost-2-gobig-0034.png"███████████████████████████████████████████████████████████████| 12

data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████|
Sampling: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████|
Starting render!
Global seed set to 9422632
Sampling:   0%|
Prompt for this image:
   ['A cute ghost seen lost, deep in the woods at night, studio lighting, dramatic, 4k uhd, fantasy, depth of field, in landsca
Parkes,  Mikalojus Konstantinas Čiurlionis, and Daniel Merriam']

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12

Output saved as "./out/ghost-2-gobig/ghost-2-gobig-0034.png"███████████████████████████████████████████████████████████████| 12

data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████|
Sampling: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████|
Starting render!
Global seed set to 9422633
Sampling:   0%|
Prompt for this image:
   ["A cute ghost seen lost, deep in the woods at night, specular lighting, noxious, embellishments, victorian, oil painting, i
by Chris Foss,  O'Keeffe, and Jean-Michel Basquiat"]

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12

data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████|
Sampling: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████|
Starting render!
Global seed set to 9422637
Sampling:   0%|
Prompt for this image:
   ['A cute ghost seen lost, deep in the woods at night, indirect light, biomechanical, spirals, gothic, Ukiyo-e, in lush veget
beth Shippen Green,  Chesley Bonestell, and Camille Pissarro']

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12

Output saved as "./out/ghost-2-gobig/ghost-2-gobig-0034.png"███████████████████████████████████████████████████████████████| 12

data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████|
Sampling: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████|
result is ./out/ghost-2-gobig/ghost-2-gobig-0034.png
Gobig output saved as ./out/ghost-2-gobig/ghost-2-gobig_gobig-0034.png

Another example

[KnoBuddy]

So I think if you use the frozen seed option it will freeze the seed for each slice of the gobig operation, however the way that PRS slices up the images and rediffuses them, it is going to try to create everything you put in the prompt into each slice. If you want coherence, you should create images, then use them as an init image for gobig with a different prompt. Possibly adjusting the prompt for each individual slice manually.

Currently there are no automated processes to do such a thing. The closest was the JAX/HUEMIN Collab notebook, but I never even got that working properly.

[KnoBuddy]

If you want it to do a new image and go big for each image, you can try using the from_file option to specify a text file.

For each line in the text file it will run N batches and N iters, as well as run GOBig after each one. (At least I think that's how it works. I don't know that I've tested it but it's how I expected it to work when writing the GUI.)

[rockerBOO]

Thanks for the feedback @KnoBuddy . Some of these could be workarounds but not sure if any of them actually would work. Even doing 1 batch and 1 iters it still recalculates the prompt (im using placeholders).

Keeping a static prompt or using a init_image could work to make it bigger with 1.0 init_strength could work for now. But could just use the realesrgan (set scale) separately to do something like that?

I was considering how gobig was working and I can somewhat see how the seed wouldn't be a big deal but I think changing the prompt (when using randomizer) is a mistake.

The batches/iter only do gobig on the last one or an additional run (I'd have to clarify) and not for each created image. I feel this makes it difficult to work with both. I can probably look into the logic and resolve this factor if it's desired.

[KnoBuddy]

Use from_file to perform gobig at the end of each prompt. Each batch is run n_iter times for each line of the file specified in from_file in the json

[rockerBOO]

I feel this is all probably as intended, and there's a large section of the README that details some of these concerns at this current time. I will consider some of these changes to workaround, as well as possibly making it more flexible and more consistent in how it works.