search

AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
139.66k stars 26.47k forks source link

[Bug]: When running X/Y/Z plot where one of the axis is checkpoint, UI selected checkpoint reloads after each image before reloading the script selected checkpoint again to create the next image #12641

Closed Alyndiar closed 1 year ago

Alyndiar commented 1 year ago

Is there an existing issue for this?

What happened?

I am using X/Y/Z plot script. X axis is a list of checkpoints. I use X axis to minimize the loading of models as this slows generation. For each image produced where the checkpoint selected by X/Y/Z plot is different from the checkpoint selected in the UI itself, the new model loads, 1 image is created then, the UI selected checkpoint is reloaded before the loop continues. The one saving grace is that if you don't use the new method to manage multiple models loaded and instead use the old (obsolete) number of models cached, the VAE and optimization (ie. xformers) don't reload each time. If you do use the new methods of managing multiple models and set number of models cached as suggested, VAE and optimizers are reapplied each time.

Steps to reproduce the problem

  1. Activate X/Y/Z plot in the scripts section
  2. Use Checkpoint as one of the axis of the plot
  3. Select 2 or more model that are different from the model selected in the UI
  4. Add something to Y axis to generate multiple images
  5. Click Generate

What should have happened?

When using X/Y/Z plot, the model selected in the main UI should not be reloaded after each image is generated. It should be reloaded ONLY once the X/Y/Z plot script is complete or interrupted, just before making the generate button available again.

Version or Commit where the problem happens

Version: v1.5.1-495-g541ef924 Commit hash: 541ef9247cfe8785c12f7142ad02cd5310fc5925

What Python version are you running on ?

Python 3.10.x

What platforms do you use to access the UI ?

Windows

What device are you running WebUI on?

Nvidia GPUs (RTX 20 above)

Cross attention optimization

xformers

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--api --no-half-vae --theme dark --xformers --administrator --upcast-sampling --opt-channelslast --update-check --update-all-extensions --vae-dir E:\AI\models\VAE

List of extensions

Extension URL Branch Version Date Update CFG-Schedule-for-Automatic1111-SD https://github.com/guzuligo/CFG-Schedule-for-Automatic1111-SD.git main 84892c88 Thu Mar 9 10:43:02 2023 latest CloneCleaner https://github.com/artyfacialintelagent/CloneCleaner.git main 76348e1a Mon Jun 26 16:42:33 2023 latest StyleSelectorXL https://github.com/ahgsql/StyleSelectorXL.git main d7abf002 Thu Aug 10 00:23:36 2023 latest Styles-Editor https://github.com/chrisgoringe/Styles-Editor.git main bb0b51b6 Tue Aug 1 22:12:09 2023 latest a1111-sd-webui-tagcomplete https://github.com/DominikDoom/a1111-sd-webui-tagcomplete.git main 5deb72cd Wed Aug 16 08:50:05 2023 latest adetailer https://github.com/Bing-su/adetailer.git main 22602536 Mon Aug 14 14:45:42 2023 latest latent-upscale https://github.com/feynlee/latent-upscale.git main b9f75f44 Mon Jul 17 21:55:32 2023 latest loopback_scaler https://github.com/Elldreth/loopback_scaler.git main 184a53e6 Fri Jun 23 21:42:22 2023 latest model-keyword https://github.com/mix1009/model-keyword.git main e433eaf7 Fri Aug 18 06:00:32 2023 latest multidiffusion-upscaler-for-automatic1111 https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111.git main f9f8073e Sat Jul 22 06:53:11 2023 latest sd-civitai-browser https://github.com/SignalFlagZ/sd-civitai-browser.git mod aa109f1f Thu Aug 17 11:36:25 2023 latest sd-dynamic-prompts https://github.com/adieyal/sd-dynamic-prompts.git main 2917434e Mon Aug 14 09:50:14 2023 latest sd-history-slider https://github.com/LucasMali/sd-history-slider main b020b626 Tue Jul 18 17:50:51 2023 latest sd-infinity-grid-generator-script https://github.com/mcmonkeyprojects/sd-infinity-grid-generator-script.git master 944c71f5 Wed Aug 2 06:32:50 2023 latest sd-webui-animatediff https://github.com/continue-revolution/sd-webui-animatediff.git master 19191b55 Tue Aug 8 10:11:38 2023 latest sd-webui-aspect-ratio-helper https://github.com/thomasasfk/sd-webui-aspect-ratio-helper.git main 99fcf9b0 Sun Jun 4 15:39:07 2023 latest sd-webui-color-enhance https://git.mmaker.moe/mmaker/sd-webui-color-enhance.git master b9e4a2cf Sat Jun 10 18:52:53 2023 latest sd-webui-controlnet https://github.com/Mikubill/sd-webui-controlnet.git main 0cfc88b6 Mon Aug 14 00:07:03 2023 latest sd-webui-inpaint-anything https://github.com/Uminosachi/sd-webui-inpaint-anything.git main 76818e7d Thu Aug 17 10:55:12 2023 latest sd-webui-llul https://github.com/hnmr293/sd-webui-llul.git master aa47b3ee Thu May 4 16:14:34 2023 latest sd-webui-prompt-history https://github.com/namkazt/sd-webui-prompt-history main bde3e95e Sat Jul 15 06:23:45 2023 latest sd-webui-refiner https://github.com/wcde/sd-webui-refiner.git main 0ad12e75 Sat Aug 12 18:07:55 2023 latest sd-webui-segment-anything https://github.com/continue-revolution/sd-webui-segment-anything.git master 62ccb019 Fri Aug 11 07:43:14 2023 latest sd-webui-stablesr https://github.com/pkuliyi2015/sd-webui-stablesr.git master 4499d796 Sat Jul 1 16:38:32 2023 latest sd-webui-supermerger https://github.com/hako-mikan/sd-webui-supermerger.git main 8d409674 Tue Aug 8 20:47:34 2023 behind HEAD sd-webui-vectorscope-cc https://github.com/Haoming02/sd-webui-vectorscope-cc.git main 245c209a Tue Aug 15 05:37:32 2023 latest sd_civitai_extension https://github.com/civitai/sd_civitai_extension main 6afadb00 Fri Aug 4 23:08:54 2023 latest sd_shutdown_button https://github.com/EnsignMK/sd_shutdown_button.git main c11b5ab2 Thu Aug 17 18:58:36 2023 latest sd_web_ui_preset_utils https://github.com/Gerschel/sd_web_ui_preset_utils.git master 6e5648df Sun May 28 20:37:30 2023 latest stable-diffusion-webui-randomize https://github.com/innightwolfsleep/stable-diffusion-webui-randomize.git master c04c7611 Tue Jan 31 14:54:33 2023 latest stable-diffusion-webui-state https://github.com/ilian6806/stable-diffusion-webui-state.git main f2bb3809 Thu Jul 20 06:12:49 2023 latest stable-diffusion-webui-wd14-tagger https://github.com/toriato/stable-diffusion-webui-wd14-tagger.git master 99bf7d81 Mon Jul 17 17:44:24 2023 latest ultimate-upscale-for-automatic1111 https://github.com/Coyote-A/ultimate-upscale-for-automatic1111.git master c99f382b Tue Jun 13 04:29:35 2023 latest LDSR built-in None Fri Aug 18 06:40:26 2023
Lora built-in None Fri Aug 18 06:40:26 2023
ScuNET built-in None Fri Aug 18 06:40:26 2023
SwinIR built-in None Fri Aug 18 06:40:26 2023
canvas-zoom-and-pan built-in None Fri Aug 18 06:40:26 2023
extra-options-section built-in None Fri Aug 18 06:40:26 2023
mobile built-in None Fri Aug 18 06:40:26 2023
prompt-bracket-checker built-in None Fri Aug 18 06:40:26 2023

Console logs

Here is an example with the new method:
Reusing loaded model anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f] to load aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]
Loading weights [25ba966c5d] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 2.8s (send model to cpu: 1.7s, apply weights to model: 0.3s, load VAE: 0.1s, move model to device: 0.5s).
Reusing loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d] to load anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f]
Loading weights [ef49fbb25f] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 1.9s (send model to cpu: 0.9s, apply weights to model: 0.3s, load VAE: 0.1s, move model to device: 0.5s).
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00,  9.78it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.96it/s]
Reusing loaded model anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f] to load aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]
Loading weights [25ba966c5d] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 3.0s (send model to cpu: 1.8s, apply weights to model: 0.4s, load VAE: 0.1s, move model to device: 0.5s).
Reusing loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d] to load anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f]
Loading weights [ef49fbb25f] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 1.9s (send model to cpu: 1.0s, apply weights to model: 0.3s, move model to device: 0.5s).
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00,  9.56it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.97it/s]
Reusing loaded model anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f] to load aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]
Loading weights [25ba966c5d] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 2.9s (send model to cpu: 1.8s, apply weights to model: 0.4s, load VAE: 0.1s, move model to device: 0.5s).
Reusing loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d] to load anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f]
Loading weights [ef49fbb25f] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 2.3s (send model to cpu: 1.1s, apply weights to model: 0.5s, load VAE: 0.2s, move model to device: 0.5s).
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00,  9.31it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  5.05it/s]
Reusing loaded model anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f] to load aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]
Loading weights [25ba966c5d] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 2.8s (send model to cpu: 1.7s, apply weights to model: 0.3s, load VAE: 0.1s, move model to device: 0.5s).
Reusing loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d] to load anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f]
Loading weights [ef49fbb25f] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 1.9s (send model to cpu: 0.9s, apply weights to model: 0.3s, load VAE: 0.1s, move model to device: 0.5s).
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00,  9.74it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  5.01it/s]
Reusing loaded model anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f] to load aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]
Loading weights [25ba966c5d] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 2.8s (send model to cpu: 1.7s, apply weights to model: 0.4s, load VAE: 0.1s, move model to device: 0.5s).
Reusing loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d] to load anyloraCheckpoint_bakedvaeBlessedFp16.safetensors [ef49fbb25f]
Loading weights [ef49fbb25f] from cache
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 1.8s (send model to cpu: 0.9s, apply weights to model: 0.3s, move model to device: 0.5s).
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00,  9.59it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  5.04it/s]

Here is an example with the old method:
Reusing loaded model comicsArthemy_v30.safetensors [e0aa270060] to load analogMadness_v50.safetensors [f968fc436a]
Loading weights [f968fc436a] from W:\seait\stable-diffusion-webui\models\Stable-diffusion\analogMadness_v50.safetensors
Loading VAE weights specified in settings: cached customvae-mse-84000.pt
Applying attention optimization: xformers... done.
Weights loaded in 37.6s (calculate hash: 1.5s, load weights from disk: 4.5s, apply weights to model: 30.7s, load VAE: 0.4s, move model to device: 0.5s).
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00,  9.61it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.99it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:20<00:00,  1.75it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 14.32it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.97it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.70it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 14.41it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.98it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:20<00:00,  1.73it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 12.23it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  5.01it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.67it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 11.69it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.96it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.70it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 13.96it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.98it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.70it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 12.09it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  5.01it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.69it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 14.45it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.87it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.71it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 14.17it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  5.02it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:20<00:00,  1.72it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 14.21it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  4.41it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.70it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 14.24it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  5.02it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.68it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s
100%|████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 14.20it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 13/13 [00:02<00:00,  5.01it/s]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████| 36/36 [00:21<00:00,  1.69it/s]
Using already loaded model aZovyaRPGArtistTools_v3.safetensors [25ba966c5d]: done in 0.0s
Using already loaded model analogMadness_v50.safetensors [f968fc436a]: done in 0.0s

Additional information

No response

Alyndiar commented 1 year ago

I noticed that when I was using the new method, I also had the checkbox to keep only 1 checkpoint on device at a time. More testing would be needed to ensure this was not at least partially the cause of some of the issues I reported.

Alyndiar commented 1 year ago

I did a quick check. Both the new and the old method of handling multiple models give the same result, reloading the model from the main UI after the generation of each image if different from the one being used by X/Y/Z plot. However, for both methods, only selecting "Only keep one model on device" checkbox will force the reloading of VAE and optimizers each time. Hope this clarifies the issue and makes it easier to solve.

catboxanon commented 1 year ago

This shouldn't even happen because there was logic added to the XY script to handle this months ago. https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/6803

Did a git bisect, looks like it was a regression as part of c8d453e91561e57fbb76119f688a5eae9d1a6aef @w-e-w

Edit: I take that back, shouldn't have just bisected a single file. I'm guessing the model loading rework broke this.

w-e-w commented 1 year ago

I'm looking at this problem right now

w-e-w commented 1 year ago

yeah it should have because my fine changes to bring back CSV (did quick tested by reverting my PR) my current suspicion is that this is caused by AUTO fixing stuff

w-e-w commented 1 year ago

I think I found a cause it is caused by refined

catboxanon commented 1 year ago

Yep, f1975b0213f5be400889ec04b3891d1cb571fe20 is the actual cause.

w-e-w commented 1 year ago

Auto broke my override logic

w-e-w commented 1 year ago

made a PR https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12644 currently testing

w-e-w commented 1 year ago

PR has been merged in to dev