[Bug]: Some LoRAs do not load when using sd-webui-forge

Vektor8298 commented 8 months ago

Checklist

[X] The issue exists after disabling all extensions
[X] The issue exists on a clean installation of webui
[ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[X] The issue exists in the current version of the webui
[ ] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet

What happened?

When trying to load a specific LoRA (Add More Details - Detail Enhancer / Tweaker), it does not get loaded, the console does not show the LoRA being loaded nor the final image reflects it.

Steps to reproduce the problem

Add the magic word to enable the LoRA
Generate an image
Check results to see the LoRA was not applied

What should have happened?

LoRA should be loaded and applied to the generation.

What browsers do you use to access the UI ?

Mozilla Firefox, Google Chrome, Other

Sysinfo

sysinfo-2024-02-18-02-04.json

Console logs

`glibc version is 2.39
Check TCMalloc: libtcmalloc_minimal.so.4
libtcmalloc_minimal.so.4 is linked with libc.so,execute LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so.4
Python 3.10.6 (main, Feb  9 2024, 20:12:38) [GCC 13.2.1 20230801]
Version: f0.0.14v1.8.0rc-latest-184-g43c9e3b5
Commit hash: 43c9e3b5ce1642073c7a9684e36b45489eeb4a49
Launching Web UI with arguments: --listen --always-offload-from-vram --attention-quad
amdgpu.ids: No such file or directory
Total VRAM 8176 MB, total RAM 15873 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 AMD Radeon Graphics : native
VAE dtype: torch.float32
Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --attention-split
ControlNet preprocessor location: /mnt/ts512/stable-diffusion-webui-forge/models/ControlNetPreprocessor
Loading weights [386a466799] from /mnt/ts512/stable-diffusion-webui-forge/models/Stable-diffusion/based69_v10.safetensors
model_type EPS
UNet ADM Dimension 0
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
Couldn't find VAE named WD1-4-kl-f8-anime2-bless09.safetensors; using None instead
To load target model SD1ClipModel
Begin to load 1 model
Moving model(s) has taken 0.15 seconds
Model loaded in 6.0s (load weights from disk: 2.6s, forge load real models: 2.7s, calculate empty prompt: 0.7s).
[nltk_data] Downloading package wordnet to /home/vektor/nltk_data...
[nltk_data]   Package wordnet is already up-to-date!
[nltk_data] Downloading package omw-1.4 to /home/vektor/nltk_data...
[nltk_data]   Package omw-1.4 is already up-to-date!
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
[-] ADetailer initialized. version: 24.1.2, num models: 14
2024-02-17 23:19:50,192 - ControlNet - INFO - ControlNet UI callback registered.
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 25.7s (prepare environment: 4.7s, import torch: 4.2s, import gradio: 1.0s, setup paths: 1.3s, other imports: 0.8s, load scripts: 11.7s, create ui: 1.6s, gradio launch: 0.3s).
To load target model SD1ClipModel
Begin to load 1 model
unload clone 0
Moving model(s) has taken 0.21 seconds
To load target model BaseModel
Begin to load 1 model
Moving model(s) has taken 0.55 seconds
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:19<00:00,  1.57it/s]
To load target model AutoencoderKL███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:18<00:00,  1.61it/s]
Begin to load 1 model
Moving model(s) has taken 0.38 seconds

0: 640x448 1 face, 10.3ms
Speed: 9.3ms preprocess, 10.3ms inference, 77.7ms postprocess per image at shape (1, 3, 640, 448)
To load target model SD1ClipModel
Begin to load 1 model
Moving model(s) has taken 0.16 seconds
To load target model BaseModel
Begin to load 1 model
Moving model(s) has taken 0.40 seconds
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:08<00:00,  1.58it/s]
To load target model AutoencoderKL
Begin to load 1 model
Moving model(s) has taken 0.25 seconds
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:29<00:00,  1.01it/s]
Loading VAE weights specified in settings: /mnt/ts512/stable-diffusion-webui-forge/models/VAE/WD1-4-kl-f8-anime2-bless09.safetensors█████████████████████████████████████████████████| 30/30 [00:29<00:00,  1.61it/s]
VAE weights loaded.`

Additional information

GPU Driver updated recently, Arch Linux, Python 3.10.6

rerere14 commented 8 months ago

same problem

magejosh commented 8 months ago

Same problem here. It's strange though, seems like some Lora do not work at all, some just destroy the generation, and some work as expected. Problem is they all work as expected in comfyui and in a1111.

IMT-AI commented 8 months ago

I am also facing the same issue... some LORAs will be applied, and some won't. I still cannot figure out what makes the LORAs be reflected on the generated image and what not...

KNX-PT-6 commented 8 months ago

So far, have you checked the Lora button in WebUI to see if your Lora is showing up there? After some investigation, it seems that all the .safetensors Lora cannot be found by the Lora picker, and I am guessing that's what makes some Lora not working. .ckpt Loras can still be found and are shown in the picker; I assume those are probably those Lora that work?

Nerogante commented 8 months ago

My Loras were not being detected when setting set A1111_HOME=C:\SD-1.6 But they appeared after changing \ to / set A1111_HOME=C:/SD-1.6

wardensc2 commented 8 months ago

I have same problem, remove those Addetail Lora, everything work fine

KNX-PT-6 commented 8 months ago

So far, have you checked the Lora button in WebUI to see if your Lora is showing up there? After some investigation, it seems that all the .safetensors Lora cannot be found by the Lora picker, and I am guessing that's what makes some Lora not working. .ckpt Loras can still be found and are shown in the picker; I assume those are probably those Lora that work?

cough um, it seems that it's just those Loras, in my case, are not compatible with XL models. I switched back to 1.5 models and they all appeared once again. I have no clue why that's the case, but my problem was solved. yay...?

dvelm commented 8 months ago

I have same problem, remove those Addetail Lora, everything work fine

What should be removed?

vamzii commented 8 months ago

if the lora isnt available in the models directory and is still called from prompt, it doesnt show that the lora isnt available as shown in the official automatic1111 release, it generates image which is totally out of line

IMT-AI commented 8 months ago

My issue is intermittent... Sometimes it gets applied, sometimes not... I imagine a command must "block" the application of the loras if it is ran before... but I still cannot figure it out.

dvelm commented 8 months ago

Fooocus read and use the lora in the folder but this not

IMT-AI commented 8 months ago

Just updating, today I found ONE of the reasons. I used PNG Info to get the prompt from an image and tried generating. It went until a LORA I didn't have and did not use the rest. So I removed the non-existing LORA from the prompt and the rest loaded correctly. It seems it is not the only thing that makes Forge "forget' to apply a LORA, but one of them.

cokimax2 commented 7 months ago

I have the same issue. In my case, some lora and many of textual inversions are not listed neither.

Sarimae commented 6 months ago

still a problem.

pioniere commented 6 months ago

The problem is still occurring.

ikun-404 commented 5 months ago

So far, have you checked the Lora button in WebUI to see if your Lora is showing up there? After some investigation, it seems that all the .safetensors Lora cannot be found by the Lora picker, and I am guessing that's what makes some Lora not working. .ckpt Loras can still be found and are shown in the picker; I assume those are probably those Lora that work?

cough um, it seems that it's just those Loras, in my case, are not compatible with XL models. I switched back to 1.5 models and they all appeared once again. I have no clue why that's the case, but my problem was solved. yay...?

Is it really Perhaps the mechanism is like this

lllyasviel / stable-diffusion-webui-forge