search

anapnoe / stable-diffusion-webui-ux

Stable Diffusion web UI UX
GNU Affero General Public License v3.0
985 stars 58 forks source link

[Bug]: Problems in loading LoRa #179

Closed Butterfly-Dragon closed 11 months ago

Butterfly-Dragon commented 1 year ago

Is there an existing issue for this?

What happened?

For some reasons if i load stable-diffusion-webui-ux with some specific models it will work but will ignore LoRas giving out weird errors.

Loading a different model (not changing the prompt) will allow me to load the LoRas and produce an image.

Selecting that previous model (not changing the prompt) will now error out

Selecting other models might or might not ignore loras, but errors will not be shown

Steps to reproduce the problem

For some reasons if i load stable-diffusion-webui-ux with some specific models it will work but will ignore LoRas giving out weird errors.

Loading a different model (not changing the prompt) will allow me to load the LoRas and produce an image.

Selecting that previous model (not changing the prompt) will now error out

Selecting other models might or might not ignore loras, but errors will not be shown

What should have happened?

If i load stable-diffusion-webui-ux with some specific models it should not ignore LoRas giving out weird errors.

Loading a different model (not changing the prompt) should not change this fact.

Selecting that previous model (not changing the prompt) should not error out

Selecting other models should not ignore loras, any errors should be shown

Version or Commit where the problem happens

v1.3.2-512-g3843d60a

What Python version are you running on ?

Python 3.10.x

What platforms do you use to access the UI ?

Windows

What device are you running WebUI on?

Nvidia GPUs (RTX 20 above)

Cross attention optimization

Automatic

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--xformers --reinstall-xformers

List of extensions

No

Console logs

venv "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\Scripts\Python.exe"
Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Version: v1.3.2-512-g3843d60a
Commit hash: 3843d60ae7a692932db9be5ca33625382be32ee9
Installing xformers
Collecting xformers==0.0.20
  Using cached xformers-0.0.20-cp310-cp310-win_amd64.whl (97.6 MB)
Installing collected packages: xformers
Successfully installed xformers-0.0.16rc425
Installing requirements
Launching Web UI with arguments: --xformers --reinstall-xformers
Loading weights [ab800444cd] from D:\Documents\GitHub\stable-diffusion-webui-ux\models\Stable-diffusion\Stability Ai 2.0 512-Base-Ema.safetensors
Creating model from config: D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\stable-diffusion-stability-ai\configs\stable-diffusion\v2-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 865.91 M params.
Applying attention optimization: xformers... done.
Textual inversion embeddings loaded(0):
Textual inversion embeddings skipped(12): bad-artist-anime, bad-artist, bad-hands-5, bad-image-v2-39000, bad_prompt_version2, EasyNegative, glossifier_v3, gloss_enhancer_v5, NG_DeepNegative_V1_4T, NG_DeepNegative_V1_64T, NG_DeepNegative_V1_75T, smokey_eyes_v1
Model loaded in 4.7s (load weights from disk: 0.6s, find config: 1.1s, create model: 0.2s, apply weights to model: 0.9s, apply half(): 0.8s, move model to device: 1.0s, calculate empty prompt: 0.2s).
preload_extensions_git_metadata for 7 extensions took 0.00s
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 11.7s (import torch: 1.6s, import gradio: 0.8s, import ldm: 1.8s, other imports: 0.8s, load scripts: 1.1s, create ui: 5.1s, gradio launch: 0.4s).
loading Lora D:\Documents\GitHub\stable-diffusion-webui-ux\models\Lora\allymech.safetensors: RuntimeError
Traceback (most recent call last):
  File "D:\Documents\GitHub\stable-diffusion-webui-ux\extensions-builtin\Lora\lora.py", line 253, in load_loras
    lora = load_lora(name, lora_on_disk)
  File "D:\Documents\GitHub\stable-diffusion-webui-ux\extensions-builtin\Lora\lora.py", line 211, in load_lora
    module.weight.copy_(weight)
RuntimeError: output with shape [128, 320] doesn't match the broadcast shape [128, 320, 128, 320]

loading Lora D:\Documents\GitHub\stable-diffusion-webui-ux\models\Lora\Aven.safetensors: RuntimeError
Traceback (most recent call last):
  File "D:\Documents\GitHub\stable-diffusion-webui-ux\extensions-builtin\Lora\lora.py", line 253, in load_loras
    lora = load_lora(name, lora_on_disk)
  File "D:\Documents\GitHub\stable-diffusion-webui-ux\extensions-builtin\Lora\lora.py", line 211, in load_lora
    module.weight.copy_(weight)
RuntimeError: output with shape [25, 320] doesn't match the broadcast shape [25, 320, 25, 320]

loading Lora D:\Documents\GitHub\stable-diffusion-webui-ux\models\Lora\Anthro_Birds-v3.safetensors: RuntimeError
Traceback (most recent call last):
  File "D:\Documents\GitHub\stable-diffusion-webui-ux\extensions-builtin\Lora\lora.py", line 253, in load_loras
    lora = load_lora(name, lora_on_disk)
  File "D:\Documents\GitHub\stable-diffusion-webui-ux\extensions-builtin\Lora\lora.py", line 211, in load_lora
    module.weight.copy_(weight)
RuntimeError: output with shape [128, 320] doesn't match the broadcast shape [128, 320, 128, 320]

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:40<00:00,  3.73it/s]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:41<00:00,  3.62it/s]
Loading weights [7f96a1a9ca] from D:\Documents\GitHub\stable-diffusion-webui-ux\models\Stable-diffusion\Anything V5.0.safetensors█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:41<00:00,  3.83it/s]
Creating model from config: D:\Documents\GitHub\stable-diffusion-webui-ux\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying attention optimization: xformers... done.
Textual inversion embeddings loaded(12): bad-artist, bad-artist-anime, bad-hands-5, bad-image-v2-39000, bad_prompt_version2, EasyNegative, gloss_enhancer_v5, glossifier_v3, NG_DeepNegative_V1_4T, NG_DeepNegative_V1_64T, NG_DeepNegative_V1_75T, smokey_eyes_v1
Model loaded in 2.4s (create model: 0.4s, apply weights to model: 0.5s, apply half(): 0.6s, move model to device: 0.9s).
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:43<00:00,  3.44it/s]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:42<00:00,  3.51it/s]
Loading weights [ab800444cd] from D:\Documents\GitHub\stable-diffusion-webui-ux\models\Stable-diffusion\Stability Ai 2.0 512-Base-Ema.safetensors█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:42<00:00,  3.54it/s]
Creating model from config: D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\stable-diffusion-stability-ai\configs\stable-diffusion\v2-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 865.91 M params.
Applying attention optimization: xformers... done.
Textual inversion embeddings loaded(0):
Textual inversion embeddings skipped(12): bad-artist-anime, bad-artist, bad-hands-5, bad-image-v2-39000, bad_prompt_version2, EasyNegative, glossifier_v3, gloss_enhancer_v5, NG_DeepNegative_V1_4T, NG_DeepNegative_V1_64T, NG_DeepNegative_V1_75T, smokey_eyes_v1
Model loaded in 4.0s (find config: 1.1s, create model: 0.3s, apply weights to model: 0.7s, apply half(): 0.7s, move model to device: 1.0s, calculate empty prompt: 0.3s).
  0%|                                                                                                                                                                                                                                                                           | 0/150 [00:00<?, ?it/s]
*** Error completing request
*** Arguments: ('task(po1qw78twzimdg3)', '', '', ['Philomela2', 'Standard Barebone', 'Philomela1'], 150, 18, False, False, 1, 1, 30, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '',[], 0, '', [], True, False, False, False, 0) {}
    Traceback (most recent call last):
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\call_queue.py", line 55, in f
        res = list(func(*args, **kwargs))
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\call_queue.py", line 35, in f
        res = func(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\txt2img.py", line 57, in txt2img
        processed = processing.process_images(p)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\processing.py", line 620, in process_images
        res = process_images_inner(p)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\processing.py", line 739, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\processing.py", line 992, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\sd_samplers_kdiffusion.py", line 433, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\sd_samplers_kdiffusion.py", line 275, in launch_sampling
        return func()
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\sd_samplers_kdiffusion.py", line 433, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\k-diffusion\k_diffusion\sampling.py", line 626, in sample_dpmpp_2m_sde
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\sd_samplers_kdiffusion.py", line 174, in forward
        x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict(c_crossattn, image_cond_in[a:b]))
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\k-diffusion\k_diffusion\external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\k-diffusion\k_diffusion\external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\sd_hijack_utils.py", line 17, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\sd_hijack_utils.py", line 28, in __call__
        return self.__orig_func(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
        x_recon = self.model(x_noisy, t, **cond)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
        out = self.diffusion_model(x, t, context=cc)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\modules\sd_unet.py", line 91, in UNetModel_forward
        return ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui(self, x, timesteps, context, *args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
        h = module(h, emb, context)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
        x = layer(x, context)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 332, in forward
        x = self.proj_in(x)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\extensions-builtin\Lora\lora.py", line 398, in lora_Linear_forward
        lora_apply_weights(self)
      File "D:\Documents\GitHub\stable-diffusion-webui-ux\extensions-builtin\Lora\lora.py", line 333, in lora_apply_weights
        self.weight += lora_calc_updown(lora, module, self.weight)
    RuntimeError: output with shape [320, 320] doesn't match the broadcast shape [320, 320, 320, 320]

---
Calculating sha256 for D:\Documents\GitHub\stable-diffusion-webui-ux\models\Stable-diffusion\Stability Ai V1-5-Pruned-Emaonly.safetensors: 6ce0161689b3853acaa03779ec93eafe75a02f4ced659bee03f50797806fa2fa
Loading weights [6ce0161689] from D:\Documents\GitHub\stable-diffusion-webui-ux\models\Stable-diffusion\Stability Ai V1-5-Pruned-Emaonly.safetensors
Creating model from config: D:\Documents\GitHub\stable-diffusion-webui-ux\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying attention optimization: xformers... done.
Textual inversion embeddings loaded(12): bad-artist, bad-artist-anime, bad-hands-5, bad-image-v2-39000, bad_prompt_version2, EasyNegative, gloss_enhancer_v5, glossifier_v3, NG_DeepNegative_V1_4T, NG_DeepNegative_V1_64T, NG_DeepNegative_V1_75T, smokey_eyes_v1
Model loaded in 2.6s (create model: 0.4s, apply weights to model: 0.6s, apply half(): 0.6s, move model to device: 0.8s, calculate empty prompt: 0.2s).
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:41<00:00,  3.58it/s]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:41<00:00,  3.64it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:39<00:00,  3.75it/s]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:40<00:00,  3.74it/s]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:40<00:00,  3.71it/s]

Additional information

Stuff i downloaded and added manually to the clean installation.

Embeddings: bad-artist-anime.pt bad-artist.pt bad-hands-5.pt bad-image-v2-39000.pt bad_prompt_version2.pt EasyNegative.pt glossifier_v3.pt gloss_enhancer_v5.pt NG_DeepNegative_V1_4T.pt NG_DeepNegative_V1_64T.pt NG_DeepNegative_V1_75T.pt smokey_eyes_v1.pt

LoRa: allymech.safetensors Anthro_Birds-v3.safetensors Assaultron.safetensors Aven.safetensors clockworkspiderV2.safetensors creature00d-000010.safetensors Cyber_Armor.safetensors cyborgbot.safetensors EdobBirds_v1.0.safetensors mechanicalarms.safetensors mecha_offset.safetensors Robotaction.safetensors RX78.safetensors

Models: Anything V5.0.safetensors D&Diffusion3.0 Protogen fp32.safetensors Divine Elegance Mix V4.safetensors Dusktide Mix V10.safetensors Furry Diffusion.ckpt Furry Vixens V20.safetensors Girl And Robot Mix V10.ckpt Identity Crisis EroDemon Mix 03 WS.safetensors Lyriel V15.safetensors Novel AI.ckpt Novel AI.yaml Pro Fantasy V22.safetensors RedShift-WD12-SD14-NAI-FMD.ckpt ReV Animated V122.safetensors Robo Diffusion V1.ckpt SD21-E621-Rising-V2-Epoch-29.ckpt SD21-E621-Rising-V2-Epoch-29.yaml Stability Ai 2.0 512-Base-Ema.safetensors Stability Ai V1-5-Pruned-Emaonly.safetensors Strange Dedication V3.safetensors Strange Dedication V4-Fp16.safetensors Trinart Characters IT4 V1.ckpt Waifu Diffusion 1-4-Anime E2.ckpt Yiffy E18.ckpt Zack3D Kinky V1.ckpt

Butterfly-Dragon commented 1 year ago

just to make sure... here is me reinstalling this from scratch and doing the exact same thing above Stable Diffusion WebUI.txt

FutonGama commented 1 year ago

wait... how do you knows it is related to that?!

WTF, sorry, i commented in the wrong post, i was two things opened and by mistake i commented here.