Closed Woisek closed 1 year ago
this is a bit strange that lycoris plugin is getting triggered during embedding training. just as a test, can you to to extensions and uncheck lycoris extension, restart server (not just restart from ui) and try training again? if it works, its going to be much quicker to nail this down.
btw, i just tried quick embedding training and could not reproduce.
I'm preparing a new training right now, I will report back.
please update the repo before training, i've added some additional logging.
So, now I get this:
Created embedding: I:\Super SD 2.0\automatic\models\embeddings\MarieMikulecka.pt vectors 8 init woman Preparing dataset... 100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:03<00:00, 7.47it/s] No saved optimizer exists in checkpoint 100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 5.05it/s] Loading CLiP model ViT-L/14 ███████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 5.11it/s] embedding train: TypeError ┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐ │ I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py:599 in train_embedding │ │ │ │ 598 │ │ │ │ │ │ captioned_image = caption_image_overlay(image, title, footer_lef │ │ > 599 │ │ │ │ │ │ captioned_image = insert_image_data_embed(captioned_image, data) │ │ 600 │ │ │ │ I:\Super SD 2.0\automatic\modules\textual_inversion\image_embedding.py:74 in insert_image_data_embed │ │ │ │ 73 │ │ │ > 74 │ h = image.size[1] │ │ 75 │ next_size = data_np_low.shape[0] + (h-(data_np_low.shape[0] % h)) │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ TypeError: 'int' object is not subscriptable Applying scaled dot product cross attention optimization
I just happened across this issue as I was trying to solve my own RuntimeError: a leaf Variable that requires grad is being used in an in-place operation
problem. Vladmandic's comment about the Lycoris plugin being triggered made me realize mine was being caused by a lora weight being applied(I had a similar output message instead mentioning lora instead of lycoris). In my case, temporarily removing the Lora folder in extensions-builtin
, followed by restarting the ui solved my problem
Update: The TI training is starting, but shortly after breaks with a COOM. I never had a COOM while doing a TI training. With the same settings and train data on the last available A1111 version the training went perfectly.
@Woisek I assume COOM means Cuda out-of-memory? If so, can you post a traceback showing where does OOM actually happen?
I assume COOM means Cuda out-of-memory?
Yes, correct. Will do. But as another update: I restarted WebUI in the meantime and trained another TI. For now, no COOM came up. Maybe just a hiccup or something ... 🤪 But another strange thing came up, it looked like it has something to do with the Self Attention Guidance extension. But I couldn't catch it. It gave me a lot of running error messages every time an embedding preview was created. Will watch out for it and report then.
@Woisek any updated on this issue?
Haven't got this lately, but I will watch out for it.
ok, i'll keep it open.
closing for now, can be reopened if there is reoccrence and new information is provided.
An error occurs again when an embedding training is started:
Preparing dataset... 100%|██████████████████████████████████████████████████████████████████████████████████| 29/29 [00:04<00:00, 6.75it/s] Training settings file: textual_inversion\ChloePollock\settings-2023-05-22_08-42-10.json 08:42:10-167659 INFO No saved optimizer exists in checkpoint 0% ----------------------------------------------------------------------------- 0/1 [ 0:00:04 < -:--:-- , ? it/s ]08:42:14-572112 ERROR embedding train: AssertionError ──────────────────────────────────────── 0% Traceback -----------------------------------------------------------------------------(most recent call last) 0/1 ──────────────────────────────────────── [─┐ 0:00:04│ < -:--:--I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py ,: 497? it/s in ]train_embedding │ │ │ │ 496 │ │ │ │ │ clip_grad(embedding.vec, clip_grad_sched.learn_rate) │ │ > 497 │ │ │ │ scaler.step(optimizer) │ │ 498 │ │ │ │ scaler.update() │ │ │ │ I:\Super SD 2.0\automatic\venv\lib\site-packages\torch\cuda\amp\grad_scaler.py:368 in step │ │ │ │ 367 │ │ │ │ > 368 │ │ assert len(optimizer_state["found_inf_per_device"]) > 0, "No inf checks were rec │ │ 369 │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ AssertionError: No inf checks were recorded for this optimizer. 0% ----------------------------------------------------------------------------- 0/1 [ 0:00:04 < -:--:-- , ? it/s ] 08:42:14-734676 INFO Applying scaled dot product cross attention optimization
Version: d647bb5c Sun May 21 18:49:05 2023 -0400
that is different - you had:
in order to re-use embedding that is already created, that embedding must have previous optimizer state enabled. or you need to re-create embedding and set overwrite so its set to empty.
i'll improve error messages as the traceback above looks really ugly.
This happens when I want to train a completely new embedding and I always create with "Override old embedding" ...
Then there is another issue, I'll check.
Thanks!
this is actually deeeeep inside torch, not an app issue.
seems like optimizer internal states did not record and inf checks which is invalid state for optimizer looking at torch docs https://pytorch.org/docs/stable/notes/amp_examples.html and walking through torch code, it seems that can happen if autocast is not doing its job which is rare but possible
and its in 99% of cases, gpu ralated so what's your backend and gpu? post first 10 lines of your startup console log
and can you try with ui -> settings -> compute settings -> precision type -> full
15:58:49-451976 INFO Starting SD.Next 15:58:49-457962 INFO Python 3.10.6 on Windows 15:58:49-536775 INFO Version: a64bb437 Mon May 22 10:50:59 2023 -0400 15:58:50-002736 INFO Setting environment tuning 15:58:50-005728 INFO nVidia CUDA toolkit detected 15:58:53-270561 INFO Torch 2.0.0+cu118 15:58:53-289511 INFO Torch backend: nVidia CUDA 11.8 cuDNN 8700 15:58:53-292504 INFO Torch detected GPU: NVIDIA GeForce RTX 2070 SUPER VRAM 8192 Arch (7, 5) Cores 40 15:58:53-294499 INFO Verifying requirements 15:58:53-310456 WARNING Package wrong version: gradio 3.31.0 required 3.32.0 15:58:53-311454 INFO Installing package: gradio==3.32.0 15:59:05-943844 INFO Installing packages 15:59:06-037597 INFO Running setup
So this is torch related ... ? I guess then it will be a problem for more users than me ... ?!
and can you try with ui -> settings -> compute settings -> precision type -> full
And then? Try a new embedding training ... ?
So this is torch related ... ? I guess then it will be a problem for more users than me ... ?!
its specific to some cards, you have rtx2070 which is known to have flaky fp16 support in general, so quite possible that autocast messes things up when card does bad math in fp16. for cards that fail fp16 test i wrote, i automatically set correct params. but this gpu is borderline - it passes test, but can result in some strange math results sometimes. e.g. you may run generate 50 times and get 2-3 black images out of 50.
And then? Try a new embedding training ... ?
yes, set precision to full, restart server, try new embedding.
Still an error occurs, here is the full console from start:
Already up to date. Using VENV: I:\Super SD 2.0\automatic\venv 17:06:14-138919 INFO Starting SD.Next 17:06:14-158861 INFO Python 3.10.6 on Windows 17:06:15-541058 INFO Version: a64bb437 Mon May 22 10:50:59 2023 -0400 17:06:15-794380 INFO Setting environment tuning 17:06:15-798370 INFO nVidia CUDA toolkit detected 17:06:18-537238 INFO Torch 2.0.0+cu118 17:06:18-563168 INFO Torch backend: nVidia CUDA 11.8 cuDNN 8700 17:06:18-565163 INFO Torch detected GPU: NVIDIA GeForce RTX 2070 SUPER VRAM 8192 Arch (7, 5) Cores 40 17:06:18-567157 INFO Verifying requirements 17:06:18-585109 INFO Installing packages 17:06:18-681851 INFO Running setup 17:06:18-682848 INFO Installing repositories 17:06:18-851398 INFO Installing submodules 17:06:22-862416 INFO Updating submodules 17:07:11-949767 INFO Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'multidiffusion-upscaler-for-automatic1111', 'prompt-bracket-checker', 'ScuNET', 'sd-dynamic-thresholding', 'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info', 'sd-webui-additional-networks', 'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR', 'a1111-sd-webui-tagcomplete', 'adetailer', 'Auto-Photoshop-StableDiffusion-Plugin', 'canvas-zoom', 'OneButtonPrompt', 'openpose-editor', 'sd-dynamic-prompts', 'sd-model-preview-xd', 'sd-webui-ar', 'sd-webui-regional-prompter', 'sd_extension-prompt_formatter', 'sd_grid_add_image_number', 'sd_webui_SAG', 'sd_web_ui_preset_utils', 'stable-diffusion-webui-model-toolkit', 'stable-diffusion-webui-promptgen', 'stable-diffusion-webui-sonar', 'stable-diffusion-webui-two-shot', 'ultimate-upscale-for-automatic1111'] 17:07:11-955723 WARNING Extensions duplicates: ['multidiffusion-upscaler-for-automatic1111'] 17:07:11-956720 INFO Updating Wiki 17:07:13-198276 INFO Running extension preloading 17:07:13-200247 INFO Server arguments: ['--listen', '--insecure', '--theme=dark'] 17:07:18-520816 INFO Unloading xFormers 17:07:20-623407 INFO Libraries loaded 17:07:20-625401 INFO Using data path: I:\Super SD 2.0\automatic 17:07:20-709203 INFO Available VAEs: I:\Super SD 2.0\automatic\models\VAE 4 17:07:20-712169 INFO Skipping conflicting extension: I:\Super SD 2.0\automatic\extensions\multidiffusion-upscaler-for-automatic1111 17:07:22-313012 INFO Available models: I:\Super SD 2.0\automatic\models\Stable-diffusion 197 17:07:24-540462 INFO Setting Torch parameters: dtype=torch.float16 vae=torch.float16 unet=torch.float16 Loading weights: I:\Super SD 2.0\automatic\models\Stable-diffusion\Analog Madness\analog_madness_v4.0_SD1.5.safetenso… LatentDiffusion: Running in eps-prediction mode DiffusionWrapper has 859.52 M params. 17:07:25-378218 INFO Model created from config: I:\Super SD 2.0\automatic\configs\v1-inference.yaml Loading weights: I:\Super SD 2.0\automatic\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors -------- 0.0/33… -:--:-- MB 17:07:29-713607 INFO Applying scaled dot product cross attention optimization No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. No Image data blocks found. 17:07:31-652584 INFO Embeddings loaded: 61 ['arcimboldo-emb', 'arcimboldo-style', 'bad-hands-5', 'bad-prompt-2', 'Bob Ross-200', 'Bob Ross-2000', 'Bob Ross-800', 'Bob Ross-8000', 'CanellaSantori', 'ChloePollock', 'depthmap', 'durer-style', 'easynegative', 'FightsInTiSpa-emb', 'hrgiger-drmacabre', 'kc16-1000-sd1-5', 'kc16-2000-sd1-5', 'kc16-3000-sd1-5', 'kc16-4000-sd1-5', 'LauraWontorra', 'LindaHoschek', 'liquid-light', 'low-poly-hd-logos-icons', 'LucindaBunker', 'MarieMikulecka', 'Masaccio', 'neg-for_analog_portrait_SD1.5', 'NoomiRapace', 'piotr-jablonski', 'pjablonski-style', 'PlanIt', 'prsnAlmaZadic', 'prsnCaroKeb', 'prsnKaroEdtstad', 'prsnKaroEdtstad_v3', 'prsnKatelyn', 'prsnMarinaSirits', 'prsnOliviaWilde', 'prsnRoman_v3', 'prsnSallyRichardson', 'prsnSpencerLocke_v2', 'prsnTahnee', 'realMichaela_v1', 'realMichaela_v4', 'realMichaela_v4-3500', 'roy-lichtenstein', 'Style-Autumn', 'style-bobross-1150', 'style-bobross-1400', 'style-bobross-2850', 'style-bobross-4950', 'style-bobross-500', 'style-bobross-5900', 'Style-Empire', 'Style-Glorious', 'style-masaccio', 'style-masaccio2-8990', 'style-masaccio2-9920', 'Style-Necromancy', 'Style-Winter', 'tron-style'] 17:07:31-662529 INFO Textual inversion embeddings skipped(43): HorrorByDave, lavastyle-12000, mdjrny-ppc_SDv2, midjourney-style, Neg_Facelift768, papercutcraft-SDv2, PlanIt2, PortraitHelper, ScaryMonstersV2, VikingPunk, vray-render, waterworksv2-5000, webui, CGI_Animation-185, CGI_Animation-245, CGI_Animation, dgtl, dblx, rz-neg-foranalogportrait, rz-neg-general, GTA768, kc16-v2-5000, kc16-v3-6250, kc16-v4-5000, kc32-v4-5000, kc8-v4-5000, nartfixer, nfixer, nfixernext, neg-for_analog_portrait_SD2.1, neg-general_SD2.1, nrealfixer, embellish1, embellish2, embellish3, Cinema768-Analog, Cinema768-BW, Cinema768-Classic, Cinema768-Digital, Cinema768-SilentFilm, no_unrealistic768, Style-Winter2, TungstenDispo 17:07:31-672503 INFO Model loaded in 7.2s (load=0.3s create=0.5s apply=2.6s vae=1.1s move=0.6s embeddings=1.9s) 17:07:31-817142 INFO Model load finished: {'ram': {'used': 3.08, 'total': 63.95}, 'gpu': {'used': 3.08, 'total': 8.0}, 'retries': 0, 'oom': 0} [AddNet] Updating model hashes... 100%|█████████████████████████████████████████████████████████████████████████████| 107/107 [00:00<00:00, 26894.62it/s] [AddNet] Updating model hashes... 100%|████████████████████████████████████████████████████████████████████████████| 107/107 [00:00<00:00, 111279.58it/s] ControlNet v1.1.189 ControlNet v1.1.189 Image Browser: ImageReward is not installed, cannot be used. 2023-05-22-17:07:32 image_browser.py: debug_level: 4 [-] ADetailer initialized. version: 23.5.17, num models: 7 python_server_full_path: I:\Super SD 2.0\automatic\extensions\Auto-Photoshop-StableDiffusion-Plugin\server/python_server 17:07:36-940474 INFO Running extension preloading 17:07:37-406209 INFO Loading UI theme: name=black-orange style=Auto Create LRU cache (max_size=16) for preprocessor results. Create LRU cache (max_size=16) for preprocessor results. Running on local URL: http://0.0.0.0:7860
To create a public link, set share=True
in launch()
.
17:07:49-177717 INFO Local URL: http://localhost:7860/
17:07:49-179685 INFO API Docs: http://localhost:7860/docs
17:07:49-180682 INFO Initializing middleware
Create LRU cache (max_size=16) for preprocessor results.
17:07:49-661424 INFO Startup time: 36.4s (torch=3.9s gradio=1.4s libraries=2.1s vae=0.1s models=1.6s
codeformer=0.2s scripts=14.7s onchange=0.2s ui=7.4s launch=4.4s scripts
app_startedcallback=0.1s checkpoint=0.4s)
17:07:49-666383 INFO Memory used: 1.28 total: 63.95
Loading weights: I:\Super SD 2.0\automatic\models\Stable-diffusion\SD v1.5\v1-5-pruned(7.17GB).safetensors --- 0… -:-…
GB
17:13:11-068939 INFO Setting Torch parameters: dtype=torch.float16 vae=torch.float16 unet=torch.float16
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
17:13:11-486822 INFO Model created from config: I:\Super SD 2.0\automatic\configs\v1-inference.yaml
Loading weights: I:\Super SD 2.0\automatic\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors -------- 0.0/33… -:--:--
MB
17:13:17-954637 INFO Applying scaled dot product cross attention optimization
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
No Image data blocks found.
17:13:19-783776 INFO Model loaded in 9.3s (load=0.6s create=0.4s apply=5.0s vae=0.8s move=0.6s embeddings=1.8s)
17:13:20-002184 INFO Model load finished: {'ram': {'used': 7.27, 'total': 63.95}, 'gpu': {'used': 3.08, 'total':
8.0}, 'retries': 0, 'oom': 0}
17:13:43-982442 INFO Created embedding: I:\Super SD 2.0\automatic\models\embeddings\ChloePollock.pt vectors 8 init
woman
Preparing dataset...
0%| | 0/29 [00:00<?, ?it/s]
17:14:56-680095 INFO Applying scaled dot product cross attention optimization
17:14:56-681976 ERROR Exception: Input type (float) and bias type (struct c10::Half) should be the same
17:14:56-682974 ERROR Arguments: args=('task(hvmomgqq0gyyeni)', 'ChloePollock', '0.002:25, 0.001', 1, 1,
'T:\AI_training\workfiles\ChloePollock\processed', 'textual_inversion', 512, 512, False,
200, 'disabled', '0.1', False, 0, 'once', False, 5, 5, 'subject_filewords_double.txt', True,
True, 'an extreme closeup color photo of ChloePollock in a forest', '', 20, 0, 7, -1.0, 512,
512) kwargs={}
17:14:56-687960 ERROR gradio call: RuntimeError
┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ I:\Super SD 2.0\automatic\modules\call_queue.py:33 in f │
│ │
│ 32 │ │ │ try: │
│ > 33 │ │ │ │ res = func(*args, **kwargs) │
│ 34 │ │ │ │ progress.record_results(id_task, res) │
│ │
│ I:\Super SD 2.0\automatic\modules\textual_inversion\ui.py:29 in train_embedding │
│ │
│ 28 │ │ │
│ > 29 │ │ embedding, filename = modules.textual_inversion.textual_inversion.train_embeddin │
│ 30 │
│ │
│ ... 12 frames hidden ... │
│ │
│ I:\Super SD 2.0\automatic\venv\lib\site-packages\torch\nn\modules\conv.py:463 in forward │
│ │
│ 462 │ def forward(self, input: Tensor) -> Tensor: │
│ > 463 │ │ return self._conv_forward(input, self.weight, self.bias) │
│ 464 │
│ │
│ I:\Super SD 2.0\automatic\venv\lib\site-packages\torch\nn\modules\conv.py:459 in _conv_forward │
│ │
│ 458 │ │ │ │ │ │ │ _pair(0), self.dilation, self.groups) │
│ > 459 │ │ return F.conv2d(input, weight, bias, self.stride, │
│ 460 │ │ │ │ │ │ self.padding, self.dilation, self.groups) │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
RuntimeError: Input type (float) and bias type (struct c10::Half) should be the same
Edit: With this option enabled, I also can't render any images. 😔
i wish i had an old rtx card so i could test/reproduce, but as it is, i'm afraid all i can say is try modifying different settings in compute section - most likely: fp32 and no-half
So ... the torch update basically broke a prior working hardware like mine and made it dysfunctional ... ?!
which torch upgrade? from 1.13 to 2.0 or 2.0 to 2.0.1?
Obviously 2.0 to 2.0.1, because I could make some TIs some days ago. To be precise: Last Saturday I did my last TI.
i didn't know that. and its perfectly fine if you want to downgrade to torch 2.0
venv\Scripts\activate
pip uninstall torch torchvision torchaudio
pip install torch==2.0.0 torchvision==0.15.1 --index-url https://download.pytorch.org/whl/rocm5.4.2
app will not upgrade torch on its own if one is already installed unless you specify --reinstall
option.
That's good news. Where can I verify the torch version again?
you already are on torch 2.0.0, you never upgraded to 2.0.1:
17:06:18-537238 INFO Torch 2.0.0+cu118
so its something else that broke training if it worked recently.
do you know what was the last time it worked? if yes, go through setup.log
and find version that was used then and let me know, i'll check the differences.
I think this is the part:
2023-05-20 22:06:58,390 | INFO | I:\Super SD 2.0\automatic\installer.py | Starting SD.Next 2023-05-20 22:06:58,395 | INFO | I:\Super SD 2.0\automatic\installer.py | Python 3.10.6 on Windows 2023-05-20 22:06:58,433 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Git 2.39.0.windows.2 2023-05-20 22:06:58,469 | INFO | I:\Super SD 2.0\automatic\installer.py | Version: b6289d56 Sat May 20 13:36:27 2023 -0400 2023-05-20 22:06:58,910 | INFO | I:\Super SD 2.0\automatic\installer.py | Setting environment tuning 2023-05-20 22:06:58,911 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Torch overrides: cuda=False rocm=False ipex=False diml=False 2023-05-20 22:06:58,911 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Torch allowed: cuda=True rocm=True ipex=True diml=True 2023-05-20 22:06:58,913 | INFO | I:\Super SD 2.0\automatic\installer.py | nVidia CUDA toolkit detected 2023-05-20 22:06:59,199 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: torch 2.0.0+cu118 2023-05-20 22:06:59,200 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: torchvision 0.15.1+cu118 2023-05-20 22:07:02,215 | INFO | I:\Super SD 2.0\automatic\installer.py | Torch 2.0.0+cu118 2023-05-20 22:07:02,235 | INFO | I:\Super SD 2.0\automatic\installer.py | Torch backend: nVidia CUDA 11.8 cuDNN 8700 2023-05-20 22:07:02,238 | INFO | I:\Super SD 2.0\automatic\installer.py | Torch detected GPU: NVIDIA GeForce RTX 2070 SUPER VRAM 8192 Arch (7, 5) Cores 40 2023-05-20 22:07:02,240 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: tensorflow 2.12.0 2023-05-20 22:07:02,240 | INFO | I:\Super SD 2.0\automatic\installer.py | Verifying requirements 2023-05-20 22:07:02,242 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: addict 2.4.0 2023-05-20 22:07:02,243 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: aenum 3.1.12 2023-05-20 22:07:02,243 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: aiohttp 3.8.4 2023-05-20 22:07:02,243 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: anyio 3.6.2 2023-05-20 22:07:02,244 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: appdirs 1.4.4 2023-05-20 22:07:02,244 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: astunparse 1.6.3 2023-05-20 22:07:02,244 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: bitsandbytes 0.38.1 2023-05-20 22:07:02,245 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: blendmodes 2022 2023-05-20 22:07:02,245 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: clean-fid 0.1.35 2023-05-20 22:07:02,245 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: easydev 0.12.1 2023-05-20 22:07:02,245 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: extcolors 1.0.0 2023-05-20 22:07:02,246 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: facexlib 0.2.5 2023-05-20 22:07:02,246 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: filetype 1.2.0 2023-05-20 22:07:02,246 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: font-roboto 0.0.1 2023-05-20 22:07:02,246 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: fonts 0.0.3 2023-05-20 22:07:02,247 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: future 0.18.3 2023-05-20 22:07:02,247 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: gdown 4.7.1 2023-05-20 22:07:02,247 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: gfpgan 1.3.8 2023-05-20 22:07:02,247 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: GitPython 3.1.31 2023-05-20 22:07:02,248 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: httpcore 0.17.0 2023-05-20 22:07:02,248 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: inflection 0.5.1 2023-05-20 22:07:02,248 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: jsonmerge 1.9.0 2023-05-20 22:07:02,248 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: kornia 0.6.11 2023-05-20 22:07:02,249 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: lark 1.1.5 2023-05-20 22:07:02,249 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: lmdb 1.4.1 2023-05-20 22:07:02,249 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: lpips 0.1.4 2023-05-20 22:07:02,249 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: omegaconf 2.3.0 2023-05-20 22:07:02,249 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: open-clip-torch 2.16.0 2023-05-20 22:07:02,250 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: opencv-contrib-python 4.7.0.68 2023-05-20 22:07:02,250 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: piexif 1.1.3 2023-05-20 22:07:02,250 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: Pillow 9.5.0 2023-05-20 22:07:02,251 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: psutil 5.9.4 2023-05-20 22:07:02,251 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: pyyaml 6.0 2023-05-20 22:07:02,251 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: realesrgan 0.3.0 2023-05-20 22:07:02,251 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: resize-right 0.0.2 2023-05-20 22:07:02,252 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: rich 13.3.4 2023-05-20 22:07:02,252 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: safetensors 0.3.0 2023-05-20 22:07:02,252 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: scipy 1.10.1 2023-05-20 22:07:02,252 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: tb_nightly 2.13.0a20230413 2023-05-20 22:07:02,253 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: toml 0.10.2 2023-05-20 22:07:02,253 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: torchdiffeq 0.2.3 2023-05-20 22:07:02,253 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: torchsde 0.2.5 2023-05-20 22:07:02,253 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: voluptuous 0.13.1 2023-05-20 22:07:02,254 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: yapf 0.32.0 2023-05-20 22:07:02,254 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: scikit-image 0.20.0 2023-05-20 22:07:02,254 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: basicsr 1.4.2 2023-05-20 22:07:02,254 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: compel 1.1.5 2023-05-20 22:07:02,255 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: requests 2.30.0 2023-05-20 22:07:02,255 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: tqdm 4.65.0 2023-05-20 22:07:02,255 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: accelerate 0.18.0 2023-05-20 22:07:02,255 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: opencv-python 4.7.0.72 2023-05-20 22:07:02,256 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: diffusers 0.16.1 2023-05-20 22:07:02,256 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: einops 0.4.1 2023-05-20 22:07:02,256 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: gradio 3.31.0 2023-05-20 22:07:02,256 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: numexpr 2.8.4 2023-05-20 22:07:02,256 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: numpy 1.24.3 2023-05-20 22:07:02,257 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: numba 0.57.0 2023-05-20 22:07:02,257 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: pandas 1.5.3 2023-05-20 22:07:02,257 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: protobuf 3.20.3 2023-05-20 22:07:02,257 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: pytorch_lightning 1.9.4 2023-05-20 22:07:02,258 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: transformers 4.26.1 2023-05-20 22:07:02,258 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: timm 0.6.13 2023-05-20 22:07:02,258 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: tomesd 0.1.2 2023-05-20 22:07:02,258 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: urllib3 1.26.15 2023-05-20 22:07:02,259 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: onnxruntime 1.14.0 2023-05-20 22:07:02,259 | INFO | I:\Super SD 2.0\automatic\installer.py | Installing packages 2023-05-20 22:07:02,260 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Package version found: clip 1.0 2023-05-20 22:07:02,327 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Repository update time: Sat May 20 18:36:27 2023 2023-05-20 22:07:02,327 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Previous setup time: Sat May 20 16:13:34 2023 2023-05-20 22:07:02,328 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Wed May 17 09:19:35 2023 I:\Super SD 2.0\automatic\extensions-builtin;a1111-sd-webui-lycoris 2023-05-20 22:07:02,328 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Thu Apr 13 20:57:45 2023 I:\Super SD 2.0\automatic\extensions-builtin;clip-interrogator-ext 2023-05-20 22:07:02,328 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Fri May 5 20:09:44 2023 I:\Super SD 2.0\automatic\extensions-builtin;LDSR 2023-05-20 22:07:02,329 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Fri May 19 19:10:56 2023 I:\Super SD 2.0\automatic\extensions-builtin;Lora 2023-05-20 22:07:02,329 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:08:19 2023 I:\Super SD 2.0\automatic\extensions-builtin;multidiffusion-upscaler-for-automatic1111 2023-05-20 22:07:02,329 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Fri May 5 20:09:44 2023 I:\Super SD 2.0\automatic\extensions-builtin;prompt-bracket-checker 2023-05-20 22:07:02,329 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Fri May 5 20:09:44 2023 I:\Super SD 2.0\automatic\extensions-builtin;ScuNET 2023-05-20 22:07:02,330 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Tue May 9 10:27:26 2023 I:\Super SD 2.0\automatic\extensions-builtin;sd-dynamic-thresholding 2023-05-20 22:07:02,330 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat Apr 22 19:01:10 2023 I:\Super SD 2.0\automatic\extensions-builtin;sd-extension-aesthetic-scorer 2023-05-20 22:07:02,331 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Thu May 18 16:16:35 2023 I:\Super SD 2.0\automatic\extensions-builtin;sd-extension-steps-animation 2023-05-20 22:07:02,331 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Thu May 18 14:42:22 2023 I:\Super SD 2.0\automatic\extensions-builtin;sd-extension-system-info 2023-05-20 22:07:02,332 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:12 2023 I:\Super SD 2.0\automatic\extensions-builtin;sd-webui-additional-networks 2023-05-20 22:07:02,333 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:08:21 2023 I:\Super SD 2.0\automatic\extensions-builtin;sd-webui-controlnet 2023-05-20 22:07:02,333 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Thu Apr 13 20:57:46 2023 I:\Super SD 2.0\automatic\extensions-builtin;sd-webui-model-converter 2023-05-20 22:07:02,333 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Fri Apr 28 08:32:17 2023 I:\Super SD 2.0\automatic\extensions-builtin;seed_travel 2023-05-20 22:07:02,334 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:46 2023 I:\Super SD 2.0\automatic\extensions-builtin;stable-diffusion-webui-images-browser 2023-05-20 22:07:02,334 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Thu May 18 14:42:25 2023 I:\Super SD 2.0\automatic\extensions-builtin;stable-diffusion-webui-rembg 2023-05-20 22:07:02,335 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Fri May 5 20:09:44 2023 I:\Super SD 2.0\automatic\extensions-builtin;SwinIR 2023-05-20 22:07:02,335 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:17 2023 I:\Super SD 2.0\automatic\extensions;a1111-sd-webui-tagcomplete 2023-05-20 22:07:02,336 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:18 2023 I:\Super SD 2.0\automatic\extensions;adetailer 2023-05-20 22:07:02,337 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:19 2023 I:\Super SD 2.0\automatic\extensions;Auto-Photoshop-StableDiffusion-Plugin 2023-05-20 22:07:02,338 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:20 2023 I:\Super SD 2.0\automatic\extensions;canvas-zoom 2023-05-20 22:07:02,338 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Tue May 9 18:40:40 2023 I:\Super SD 2.0\automatic\extensions;multidiffusion-upscaler-for-automatic1111 2023-05-20 22:07:02,339 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:21 2023 I:\Super SD 2.0\automatic\extensions;OneButtonPrompt 2023-05-20 22:07:02,340 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:21 2023 I:\Super SD 2.0\automatic\extensions;openpose-editor 2023-05-20 22:07:02,341 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:22 2023 I:\Super SD 2.0\automatic\extensions;sd-dynamic-prompts 2023-05-20 22:07:02,342 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:23 2023 I:\Super SD 2.0\automatic\extensions;sd-model-preview-xd 2023-05-20 22:07:02,342 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:24 2023 I:\Super SD 2.0\automatic\extensions;sd-webui-ar 2023-05-20 22:07:02,343 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:24 2023 I:\Super SD 2.0\automatic\extensions;sd-webui-regional-prompter 2023-05-20 22:07:02,343 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:25 2023 I:\Super SD 2.0\automatic\extensions;sd_extension-prompt_formatter 2023-05-20 22:07:02,344 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:26 2023 I:\Super SD 2.0\automatic\extensions;sd_grid_add_image_number 2023-05-20 22:07:02,344 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:27 2023 I:\Super SD 2.0\automatic\extensions;sd_webui_SAG 2023-05-20 22:07:02,344 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:28 2023 I:\Super SD 2.0\automatic\extensions;sd_web_ui_preset_utils 2023-05-20 22:07:02,345 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:28 2023 I:\Super SD 2.0\automatic\extensions;stable-diffusion-webui-model-toolkit 2023-05-20 22:07:02,345 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:30 2023 I:\Super SD 2.0\automatic\extensions;stable-diffusion-webui-promptgen 2023-05-20 22:07:02,346 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:30 2023 I:\Super SD 2.0\automatic\extensions;stable-diffusion-webui-sonar 2023-05-20 22:07:02,346 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:31 2023 I:\Super SD 2.0\automatic\extensions;stable-diffusion-webui-two-shot 2023-05-20 22:07:02,346 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extension version: Sat May 20 16:13:32 2023 I:\Super SD 2.0\automatic\extensions;ultimate-upscale-for-automatic1111 2023-05-20 22:07:02,346 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Latest extensions time: Sat May 20 16:13:46 2023 2023-05-20 22:07:02,346 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Timestamps: version:1684604187 setup:1684595614 extension:1684595626 2023-05-20 22:07:02,349 | INFO | I:\Super SD 2.0\automatic\installer.py | Running setup 2023-05-20 22:07:02,351 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Args: {'config': 'I:\Super SD 2.0\automatic\config.json', 'ui_config': 'I:\Super SD 2.0\automatic\ui-config.json', 'medvram': False, 'lowvram': False, 'ckpt': None, 'vae': None, 'data_dir': 'I:\Super SD 2.0\automatic', 'models_dir': 'models', 'allow_code': False, 'share': False, 'insecure': True, 'use_cpu': [], 'listen': True, 'port': 7860, 'freeze': False, 'auth': None, 'authfile': None, 'autolaunch': False, 'api_auth': None, 'api_log': False, 'device_id': None, 'cors_origins': None, 'cors_regex': None, 'tls_keyfile': None, 'tls_certfile': None, 'tls_selfsign': None, 'server_name': None, 'no_hashing': False, 'no_download': False, 'profile': False, 'disable_queue': False, 'subpath': None, 'f': False, 'ui_settings_file': 'I:\Super SD 2.0\automatic\config.json', 'ui_config_file': 'I:\Super SD 2.0\automatic\ui-config.json', 'hide_ui_dir_config': False, 'theme': 'dark', 'disable_console_progressbars': True, 'disable_safe_unpickle': True, 'lowram': False, 'disable_extension_access': False, 'api': True, 'debug': False, 'reset': False, 'upgrade': False, 'api_only': False, 'use_ipex': False, 'use_directml': False, 'use_cuda': False, 'use_rocm': False, 'skip_update': False, 'skip_requirements': False, 'skip_extensions': True, 'skip_git': False, 'skip_torch': False, 'experimental': False, 'reinstall': False, 'test': False, 'version': False, 'ignore': False, 'safe': False} 2023-05-20 22:07:02,351 | INFO | I:\Super SD 2.0\automatic\installer.py | Installing repositories 2023-05-20 22:07:02,523 | INFO | I:\Super SD 2.0\automatic\installer.py | Installing submodules 2023-05-20 22:07:04,260 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Submodules list: b0d24ca645b6a5cb9752169691a1c6385c6fe6ae extensions-builtin/a1111-sd-webui-lycoris (heads/main) 9e6bbd9b8931bbe869a8e28e7005b0e13c2efff0 extensions-builtin/clip-interrogator-ext (heads/main) dc3f503111024bdb6c5d64d67c487f22d9ccede5 extensions-builtin/multidiffusion-upscaler-for-automatic1111 (sd-webui-v1.1.0-13-gdc3f503) c9721ab01af368940fbdcc0c660de8f43947b198 extensions-builtin/sd-dynamic-thresholding (heads/master) 79243697a23602ff4f9e441aa35fcfdc33bf0872 extensions-builtin/sd-extension-system-info (heads/main) ce2278d5bcf1801c6a2e6c3cb7bcb345c05275fe extensions-builtin/sd-webui-controlnet (heads/main) d19e28168268b0f2f50c8a5b7a4fa0a0d9b42b8c extensions-builtin/sd-webui-model-converter (heads/main) 7da8aec62bc263acd47d76ec9cabdb658b01fc91 extensions-builtin/stable-diffusion-webui-images-browser (heads/main) 657ae9f5486019a94dbe11d3560b28cccf35a0fd extensions-builtin/stable-diffusion-webui-rembg (heads/master) c924c47f374ac1b6e33e71f82948eb1853e2243f modules/lora (v0.6.4-51-gc924c47) 8b47a5349b59efac812664c5732c6e3947a6fec7 modules/lycoris (heads/main) d420606fc4f9735aa0a4d31f6e5ef18429508ab9 wiki (heads/master) 2023-05-20 22:07:05,398 | INFO | I:\Super SD 2.0\automatic\installer.py | Updating submodules 2023-05-20 22:07:13,512 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Installed packages: 241 2023-05-20 22:07:13,512 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'multidiffusion-upscaler-for-automatic1111', 'prompt-bracket-checker', 'ScuNET', 'sd-dynamic-thresholding', 'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info', 'sd-webui-additional-networks', 'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR'] 2023-05-20 22:07:24,472 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Extensions all: ['a1111-sd-webui-tagcomplete', 'adetailer', 'Auto-Photoshop-StableDiffusion-Plugin', 'canvas-zoom', 'multidiffusion-upscaler-for-automatic1111', 'OneButtonPrompt', 'openpose-editor', 'sd-dynamic-prompts', 'sd-model-preview-xd', 'sd-webui-ar', 'sd-webui-regional-prompter', 'sd_extension-prompt_formatter', 'sd_grid_add_image_number', 'sd_webui_SAG', 'sd_web_ui_preset_utils', 'stable-diffusion-webui-model-toolkit', 'stable-diffusion-webui-promptgen', 'stable-diffusion-webui-sonar', 'stable-diffusion-webui-two-shot', 'ultimate-upscale-for-automatic1111'] 2023-05-20 22:07:39,058 | INFO | I:\Super SD 2.0\automatic\installer.py | Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'multidiffusion-upscaler-for-automatic1111', 'prompt-bracket-checker', 'ScuNET', 'sd-dynamic-thresholding', 'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info', 'sd-webui-additional-networks', 'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR', 'a1111-sd-webui-tagcomplete', 'adetailer', 'Auto-Photoshop-StableDiffusion-Plugin', 'canvas-zoom', 'OneButtonPrompt', 'openpose-editor', 'sd-dynamic-prompts', 'sd-model-preview-xd', 'sd-webui-ar', 'sd-webui-regional-prompter', 'sd_extension-prompt_formatter', 'sd_grid_add_image_number', 'sd_webui_SAG', 'sd_web_ui_preset_utils', 'stable-diffusion-webui-model-toolkit', 'stable-diffusion-webui-promptgen', 'stable-diffusion-webui-sonar', 'stable-diffusion-webui-two-shot', 'ultimate-upscale-for-automatic1111'] 2023-05-20 22:07:39,067 | WARNING | I:\Super SD 2.0\automatic\installer.py | Extensions duplicates: ['multidiffusion-upscaler-for-automatic1111'] 2023-05-20 22:07:39,068 | INFO | I:\Super SD 2.0\automatic\installer.py | Updating Wiki 2023-05-20 22:07:40,531 | DEBUG | I:\Super SD 2.0\automatic\installer.py | Setup complete without errors: 1684616861 2023-05-20 22:07:40,560 | INFO | I:\Super SD 2.0\automatic\installer.py | Running extension preloading 2023-05-20 22:07:40,565 | INFO | I:\Super SD 2.0\automatic\launch.py | Server arguments: ['--skip-extensions', '--listen', '--insecure', '--theme=dark'] 2023-05-20 22:07:40,567 | DEBUG | I:\Super SD 2.0\automatic\launch.py | Starting WebUI 2023-05-20 22:07:50,330 | INFO | I:\Super SD 2.0\automatic\modules\import_hook.py | Unloading xFormers 2023-05-20 22:07:51,590 | INFO | I:\Super SD 2.0\automatic\webui.py | Libraries loaded 2023-05-20 22:07:51,593 | INFO | I:\Super SD 2.0\automatic\webui.py | Using data path: I:\Super SD 2.0\automatic 2023-05-20 22:07:51,665 | INFO | I:\Super SD 2.0\automatic\modules\sd_vae.py | Available VAEs: I:\Super SD 2.0\automatic\models\VAE 4 2023-05-20 22:07:51,668 | INFO | I:\Super SD 2.0\automatic\modules\extensions.py | Skipping conflicting extension: I:\Super SD 2.0\automatic\extensions\multidiffusion-upscaler-for-automatic1111 2023-05-20 22:07:52,132 | INFO | I:\Super SD 2.0\automatic\modules\sd_models.py | Available models: I:\Super SD 2.0\automatic\models\Stable-diffusion 193 2023-05-20 22:07:54,699 | INFO | I:\Super SD 2.0\automatic\modules\devices.py | Setting Torch parameters: dtype=torch.float16 vae=torch.float16 unet=torch.float16 2023-05-20 22:07:55,505 | INFO | I:\Super SD 2.0\automatic\modules\sd_models.py | Model created from config: I:\Super SD 2.0\automatic\configs\v1-inference.yaml 2023-05-20 22:07:57,179 | INFO | I:\Super SD 2.0\automatic\modules\sd_hijack.py | Applying scaled dot product cross attention optimization 2023-05-20 22:07:58,866 | INFO | I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py | Embeddings loaded: 66 ['AliLarter-900', 'arcimboldo-emb', 'arcimboldo-style', 'bad-hands-5', 'bad-prompt-2', 'Bob Ross-200', 'Bob Ross-2000', 'Bob Ross-800', 'Bob Ross-8000', 'CanellaSantori', 'ChloePollock', 'depthmap', 'durer-style', 'easynegative', 'FightsInTiSpa-emb', 'hrgiger-drmacabre', 'kc16-1000-sd1-5', 'kc16-2000-sd1-5', 'kc16-3000-sd1-5', 'kc16-4000-sd1-5', 'LauraWontorra', 'LindaHoschek-7800', 'liquid-light', 'low-poly-hd-logos-icons', 'LucindaBunker', 'MarieMikulecka', 'Masaccio', 'neg-for_analog_portrait_SD1.5', 'NoomiRapace', 'piotr-jablonski', 'pjablonski-style', 'PlanIt', 'prsnAliLarter_v2', 'prsnAliLarter_v3', 'prsnAlmaZadic', 'prsnAmyTeitel', 'prsnCaroKeb', 'prsnKaroEdtstad', 'prsnKaroEdtstad_v3', 'prsnKatelyn', 'prsnMarinaSirits', 'prsnMillaJovo', 'prsnOliviaWilde', 'prsnRoman_v3', 'prsnSallyRichardson', 'prsnSpencerLocke_v2', 'prsnTahnee', 'realMichaela_v1', 'realMichaela_v4', 'realMichaela_v4-3500', 'roy-lichtenstein', 'Style-Autumn', 'style-bobross-1150', 'style-bobross-1400', 'style-bobross-2850', 'style-bobross-4950', 'style-bobross-500', 'style-bobross-5900', 'Style-Empire', 'Style-Glorious', 'style-masaccio', 'style-masaccio2-8990', 'style-masaccio2-9920', 'Style-Necromancy', 'Style-Winter', 'tron-style'] 2023-05-20 22:07:58,878 | INFO | I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py | Textual inversion embeddings skipped(43): HorrorByDave, lavastyle-12000, mdjrny-ppc_SDv2, midjourney-style, Neg_Facelift768, papercutcraft-SDv2, PlanIt2, PortraitHelper, ScaryMonstersV2, VikingPunk, vray-render, waterworksv2-5000, webui, CGI_Animation-185, CGI_Animation-245, CGI_Animation, dgtl, dblx, rz-neg-foranalogportrait, rz-neg-general, GTA768, kc16-v2-5000, kc16-v3-6250, kc16-v4-5000, kc32-v4-5000, kc8-v4-5000, nartfixer, nfixer, nfixernext, neg-for_analog_portrait_SD2.1, neg-general_SD2.1, nrealfixer, embellish1, embellish2, embellish3, Cinema768-Analog, Cinema768-BW, Cinema768-Classic, Cinema768-Digital, Cinema768-SilentFilm, no_unrealistic768, Style-Winter2, TungstenDispo 2023-05-20 22:07:58,890 | INFO | I:\Super SD 2.0\automatic\modules\sd_models.py | Model loaded in 4.2s (load=0.4s create=0.4s apply=0.5s vae=0.6s move=0.5s embeddings=1.7s) 2023-05-20 22:07:59,046 | INFO | I:\Super SD 2.0\automatic\modules\sd_models.py | Model load finished: {'ram': {'used': 5.06, 'total': 63.95}, 'gpu': {'used': 3.08, 'total': 8.0}, 'retries': 0, 'oom': 0} 2023-05-20 22:08:00,201 | WARNING | I:\Super SD 2.0\automatic\extensions-builtin\stable-diffusion-webui-images-browser\scripts\image_browser.py | debug_level: 4 2023-05-20 22:08:06,160 | INFO | I:\Super SD 2.0\automatic\installer.py | Running extension preloading 2023-05-20 22:08:06,670 | INFO | I:\Super SD 2.0\automatic\modules\shared.py | Loading UI theme: name=black-orange style=Auto 2023-05-20 22:08:16,663 | INFO | I:\Super SD 2.0\automatic\webui.py | Local URL: http://localhost:7860/ 2023-05-20 22:08:16,665 | INFO | I:\Super SD 2.0\automatic\webui.py | API Docs: http://localhost:7860/docs 2023-05-20 22:08:16,666 | INFO | I:\Super SD 2.0\automatic\modules\middleware.py | Initializing middleware 2023-05-20 22:08:17,161 | INFO | I:\Super SD 2.0\automatic\webui.py | Startup time: 36.5s (torch=7.1s gradio=2.6s libraries=1.3s vae=0.1s models=0.5s codeformer=0.2s gfpgan=0.1s scripts=14.1s opts onchange=0.1s ui=5.6s start=4.4s scripts app_started_callback=0.1s checkpoint=0.4s) 2023-05-20 22:08:17,166 | INFO | I:\Super SD 2.0\automatic\launch.py | Memory used: 1.27 total: 63.95 2023-05-20 22:08:48,698 | INFO | I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py | No saved optimizer exists in checkpoint 2023-05-20 22:15:07,482 | INFO | I:\Super SD 2.0\automatic\modules\sd_hijack.py | Applying scaled dot product cross attention optimization 2023-05-20 22:17:20,529 | INFO | I:\Super SD 2.0\automatic\modules\devices.py | Setting Torch parameters: dtype=torch.float16 vae=torch.float16 unet=torch.float16 2023-05-20 22:17:20,927 | INFO | I:\Super SD 2.0\automatic\modules\sd_models.py | Model created from config: I:\Super SD 2.0\automatic\configs\v1-inference.yaml 2023-05-20 22:17:27,397 | INFO | I:\Super SD 2.0\automatic\modules\sd_hijack.py | Applying scaled dot product cross attention optimization 2023-05-20 22:17:29,117 | INFO | I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py | Embeddings loaded: 105 ['AliLarter-900', 'arcimboldo-emb', 'arcimboldo-style', 'bad-hands-5', 'bad-prompt-2', 'Bob Ross-200', 'Bob Ross-2000', 'Bob Ross-800', 'Bob Ross-8000', 'CanellaSantori', 'ChloePollock-10', 'ChloePollock-100', 'ChloePollock-105', 'ChloePollock-110', 'ChloePollock-115', 'ChloePollock-120', 'ChloePollock-125', 'ChloePollock-130', 'ChloePollock-135', 'ChloePollock-140', 'ChloePollock-145', 'ChloePollock-15', 'ChloePollock-150', 'ChloePollock-155', 'ChloePollock-160', 'ChloePollock-165', 'ChloePollock-170', 'ChloePollock-175', 'ChloePollock-180', 'ChloePollock-185', 'ChloePollock-190', 'ChloePollock-195', 'ChloePollock-20', 'ChloePollock-200', 'ChloePollock-25', 'ChloePollock-30', 'ChloePollock-35', 'ChloePollock-40', 'ChloePollock-45', 'ChloePollock-5', 'ChloePollock-50', 'ChloePollock-55', 'ChloePollock-60', 'ChloePollock-65', 'ChloePollock-70', 'ChloePollock-75', 'ChloePollock-80', 'ChloePollock-85', 'ChloePollock-90', 'ChloePollock-95', 'depthmap', 'durer-style', 'easynegative', 'FightsInTiSpa-emb', 'hrgiger-drmacabre', 'kc16-1000-sd1-5', 'kc16-2000-sd1-5', 'kc16-3000-sd1-5', 'kc16-4000-sd1-5', 'LauraWontorra', 'LindaHoschek-7800', 'liquid-light', 'low-poly-hd-logos-icons', 'LucindaBunker', 'MarieMikulecka', 'Masaccio', 'neg-for_analog_portrait_SD1.5', 'NoomiRapace', 'piotr-jablonski', 'pjablonski-style', 'PlanIt', 'prsnAliLarter_v2', 'prsnAliLarter_v3', 'prsnAlmaZadic', 'prsnAmyTeitel', 'prsnCaroKeb', 'prsnKaroEdtstad', 'prsnKaroEdtstad_v3', 'prsnKatelyn', 'prsnMarinaSirits', 'prsnMillaJovo', 'prsnOliviaWilde', 'prsnRoman_v3', 'prsnSallyRichardson', 'prsnSpencerLocke_v2', 'prsnTahnee', 'realMichaela_v1', 'realMichaela_v4', 'realMichaela_v4-3500', 'roy-lichtenstein', 'Style-Autumn', 'style-bobross-1150', 'style-bobross-1400', 'style-bobross-2850', 'style-bobross-4950', 'style-bobross-500', 'style-bobross-5900', 'Style-Empire', 'Style-Glorious', 'style-masaccio', 'style-masaccio2-8990', 'style-masaccio2-9920', 'Style-Necromancy', 'Style-Winter', 'tron-style'] 2023-05-20 22:17:29,136 | INFO | I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py | Textual inversion embeddings skipped(43): HorrorByDave, lavastyle-12000, mdjrny-ppc_SDv2, midjourney-style, Neg_Facelift768, papercutcraft-SDv2, PlanIt2, PortraitHelper, ScaryMonstersV2, VikingPunk, vray-render, waterworksv2-5000, webui, CGI_Animation-185, CGI_Animation-245, CGI_Animation, dgtl, dblx, rz-neg-foranalogportrait, rz-neg-general, GTA768, kc16-v2-5000, kc16-v3-6250, kc16-v4-5000, kc32-v4-5000, kc8-v4-5000, nartfixer, nfixer, nfixernext, neg-for_analog_portrait_SD2.1, neg-general_SD2.1, nrealfixer, embellish1, embellish2, embellish3, Cinema768-Analog, Cinema768-BW, Cinema768-Classic, Cinema768-Digital, Cinema768-SilentFilm, no_unrealistic768, Style-Winter2, TungstenDispo 2023-05-20 22:17:29,149 | INFO | I:\Super SD 2.0\automatic\modules\sd_models.py | Model loaded in 9.0s (load=0.3s create=0.4s apply=5.0s vae=0.8s move=0.6s embeddings=1.7s) 2023-05-20 22:17:29,372 | INFO | I:\Super SD 2.0\automatic\modules\sd_models.py | Model load finished: {'ram': {'used': 11.98, 'total': 63.95}, 'gpu': {'used': 3.14, 'total': 8.0}, 'retries': 0, 'oom': 0} 2023-05-20 22:19:27,498 | ERROR | I:\Super SD 2.0\automatic\modules\errors.py | gradio call: FileNotFoundError 2023-05-20 22:20:35,477 | INFO | I:\Super SD 2.0\automatic\scripts\xyz_grid.py | XYZ grid: images=40 grid=1 10x1 cells=1 steps=300 2023-05-20 22:20:35,499 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:20:55,585 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:21:12,474 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:21:29,201 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:21:45,882 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:22:02,652 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:22:19,362 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:22:36,009 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:22:52,695 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:23:09,452 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:24:47,860 | INFO | I:\Super SD 2.0\automatic\scripts\xyz_grid.py | XYZ grid: images=20 grid=1 5x1 cells=1 steps=150 2023-05-20 22:24:47,864 | WARNING | I:\Super SD 2.0\automatic\scripts\xyz_grid.py | XYZ grid: prompt S/R did not find ChloePollock-180 in prompt or negative prompt. 2023-05-20 22:24:47,867 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:25:04,775 | WARNING | I:\Super SD 2.0\automatic\scripts\xyz_grid.py | XYZ grid: prompt S/R did not find ChloePollock-180 in prompt or negative prompt. 2023-05-20 22:25:04,779 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:25:21,323 | WARNING | I:\Super SD 2.0\automatic\scripts\xyz_grid.py | XYZ grid: prompt S/R did not find ChloePollock-180 in prompt or negative prompt. 2023-05-20 22:25:21,327 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:25:37,834 | WARNING | I:\Super SD 2.0\automatic\scripts\xyz_grid.py | XYZ grid: prompt S/R did not find ChloePollock-180 in prompt or negative prompt. 2023-05-20 22:25:37,838 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:25:54,419 | WARNING | I:\Super SD 2.0\automatic\scripts\xyz_grid.py | XYZ grid: prompt S/R did not find ChloePollock-180 in prompt or negative prompt. 2023-05-20 22:25:54,424 | INFO | I:\Super SD 2.0\automatic\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py | Prompt matrix will create 4 images in a total of 1 batches. 2023-05-20 22:27:44,517 | INFO | I:\Super SD 2.0\automatic\scripts\xyz_grid.py | XYZ grid: images=20 grid=1 5x1 cells=1 steps=150
I can also add, that the same error occurs also in automatic1111 if that helps ...
you're saying that this version worked fine?
Version: https://github.com/vladmandic/automatic/commit/b6289d56c744ae9d0d16aaa1f503f0ea2ff76ec1 Sat May 20 13:36:27 2023 -0400
you can always try:
git checkout b6289d5
and to go back to latest:
git checkout master
2023-05-20 22:17:29,117 | INFO | I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py | Embeddings loaded: 105 ['AliLarter-900', 'arcimboldo-emb', 'arcimboldo-style', 'bad-hands-5', 'bad-prompt-2', 'Bob Ross-200', 'Bob Ross-2000', 'Bob Ross-800', 'Bob Ross-8000', 'CanellaSantori', 'ChloePollock-10', 'ChloePollock-100', 'ChloePollock-105', 'ChloePollock-110', 'ChloePollock-115', 'ChloePollock-120', 'ChloePollock-125', 'ChloePollock-130', 'ChloePollock-135', 'ChloePollock-140', 'ChloePollock-145', 'ChloePollock-15', 'ChloePollock-150', 'ChloePollock-155', 'ChloePollock-160', 'ChloePollock-165', 'ChloePollock-170', 'ChloePollock-175', 'ChloePollock-180', 'ChloePollock-185', 'ChloePollock-190', 'ChloePollock-195', 'ChloePollock-20', 'ChloePollock-200', 'ChloePollock-25', 'ChloePollock-30', 'ChloePollock-35', 'ChloePollock-40', 'ChloePollock-45', 'ChloePollock-5', 'ChloePollock-50', 'ChloePollock-55', 'ChloePollock-60', 'ChloePollock-65', 'ChloePollock-70', 'ChloePollock-75', 'ChloePollock-80', 'ChloePollock-85', 'ChloePollock-90', 'ChloePollock-95',
Chloe Pollock was the TI I made at this time.
I try your suggested version.
No luck. 😔 I even tried a few more days back but it stays the same. This is really strange. My ideas are out ...
perhaps its some of the packages - i know its painful, but when you go back to specific commit also delete venv
, config.json
, ui-config.json
so it reinstalls everything as-it-was at the time of the commit and uses default config? you can do that in a separate cloned copy, so you're not messing with your primary all the time.
i wish i could reproduce, then it would be easy to fix.
The thing is, it's also not working (anymore) in A1111, so I guess it must be something that both repos have i common. But what that is, is still hard to tell ...
can we try to isolate? completely fresh install (clone to a different folder), no extensions, using default sd model?
Will try probably on the weekend. I have to make a bit space, as the whole SD thing is getting really big on my drive ... 😁
Update: I just noticed, that I had made a backup from the automatic folder on mid April. Is it possible to use this folder for testing? I renamed the folder to automatic_backup, does that matter or can I just run it?
Update: So, I used the backup to create a complete new instance because the selecting of a commit didn't work. Anyway. I did a fresh install and added only the base checkpoint v1.5 to test the training and ... it worked. 😳 Sooo ... now I know that something seems to be weird in my actual installation and the big question is: How do I find out? 🤔 Any ideas on how to tackle this? 😶
instance is defined by
config.json
and ui-config.json
venv
where all packages are installedso make backups of those and then just delete them in your "prod".
But could not something in the venv cause my problem? I red often that one should delete the venv folder to let everything install correct again. 🤔
thats what i'm saying - make backup just in case you want to use it again later and then delete it so its auto-created from scratch.
Ah, OK, got it ... 👍
any updates?
Oh, sorry, hadn't got the time to make the change. 😐 I will close it for now, with the option to reopen when I made the change and know more.
Issue Description
I wanted to make an embedding, but when I click on train, this comes up:
Preparing dataset... 100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:02<00:00, 9.44it/s] No saved optimizer exists in checkpoint 0%| | 0/150 [00:00<?, ?it/s]embedding train: RuntimeError ┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐ │ I:\Super SD 2.0\automatic\modules\textual_inversion\textual_inversion.py:499 in train_embedding │ │ │ │ 498 │ │ │ │ │ else: │ │ > 499 │ │ │ │ │ │ loss = shared.sd_model.forward(x, cond)[0] / gradient_step │ │ 500 │ │ │ │ │ del x │ │ │ │ I:\Super SD 2.0\automatic\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py:846 in forward │ │ │ │ 845 │ │ │ │ c = self.q_sample(x_start=c, t=tc, noise=torch.randn_like(c.float())) │ │ > 846 │ │ return self.p_losses(x, c, t, *args, **kwargs) │ │ 847 │ │ │ │ ... 11 frames hidden ... │ │ │ │ I:\Super SD 2.0\automatic\extensions-builtin\a1111-sd-webui-lycoris....\extensions-builtin/Lora\lora.py:322 in │ │ lora_Linear_forward │ │ │ │ 321 def lora_Linear_forward(self, lora_input): │ │ > 322 │ lora_apply_weights(self) │ │ 323 │ │ │ │ I:\Super SD 2.0\automatic\extensions-builtin\a1111-sd-webui-lycoris....\extensions-builtin/Lora\lora.py:285 in │ │ lora_applyweights │ │ │ │ 284 │ │ │ else: │ │ > 285 │ │ │ │ self.weight.copy(weights_backup) │ │ 286 │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ RuntimeError: a leaf Variable that requires grad is being used in an in-place operation. Applying scaled dot product cross attention optimization
Any ideas? 😶
Version Platform Description
WIn10 Firefox 106 22:49:04-879486 INFO Version: 82d7b45c Mon Apr 24 16:54:12 2023 -0400