[Bug]: loractl Issue when activated via API

altoiddealer commented 2 days ago

Checklist

[ ] The issue exists after disabling all extensions
[ ] The issue exists on a clean installation of webui
[ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[ ] The issue exists in the current version of the webui
[ ] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet

What happened?

I'm really scratching my head as to why I'm experiencing this issue via API, but not via the UI...

Here is a json dump of my payload immediately before it is posted to txt2img endpoint:

{"prompt": "Grimace, <lora:PE_Grimace:0.75@0.0,0.38@0.5> from McDonald's, covered in green and black mold, ral-mold, <lora:ral-mold-sdxl:0.15@0.0,0.3@0.5>, with a sad expression, in a dark and damp environment, realistic and detailed illustration", "negative_prompt": "low quality, jpeg artifacts, blurry, poorly drawn, ugly, worst quality", "seed": -1, "override_settings": {}, "restore_faces": false, "sampler_name": "DPM++ 2M", "scheduler": "karras", "steps": 30, "distilled_cfg_scale": 3.5, "cfg_scale": 7, "height": 1024, "width": 1024, "enable_hr": false, "hr_upscaler": "Latent", "denoising_strength": null, "hr_scale": 2, "hr_second_pass_steps": 0, "refiner_switch_at": 0.8, "mask_blur": 4, "inpainting_fill": 1, "inpaint_full_res": true, "inpaint_full_res_padding": 32, "inpainting_mask_invert": 0, "alwayson_scripts": {"controlnet": {"args": []}, "layerdiffuse": {"args": [false, "(SDXL) Only Generate Transparent Image (Attention Injection)", 1.0, 1.0, null, null, null, "Crop and Resize", false, "", "", ""]}, "reactor": {"args": ["", false, "0", "0", "inswapper_128.onnx", "CodeFormer", 1, true, "4x_NMKD-Superscale-SP_178000_G", 1.5, 1, false, true, 1, 0, 0, false, 0.8, false, false, "CUDA", true, 0, "", "", null, true, true, 0.6, 2, null]}}}

For some reason, when used via API, the loractl parsing is capturing the entire string as "the first element".

I added a print statement in \extensions-builtin\Lora\extra_networks_lora.py

        for params in params_list:
            assert params.items

            names.append(params.positional[0])

            # THE ADDED PRINT STATEMENT
            positionalone = params.positional[1] if len(params.positional) > 1 else None
            print("POSITIONAL ONE:", positionalone)

            te_multiplier = float(params.positional[1]) if len(params.positional) > 1 else 1.0

So this is what gets printed when used via API:

POSITIONAL ONE: 0.75@0.0,0.38@0.5
activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000001B5958573A0>, <modules.extra_networks.ExtraNetworkParams object at 0x000001B4A6D125F0>]: ValueError
Traceback (most recent call last):
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\extra_networks.py", line 135, in activate
    extra_network.activate(p, extra_network_args)
  File "C:\0_SD\stable-diffusion-webui-reForge\extensions-builtin\Lora\extra_networks_lora.py", line 35, in activate
    te_multiplier = float(params.positional[1]) if len(params.positional) > 1 else 1.0
ValueError: could not convert string to float: '0.75@0.0,0.38@0.5'

If I go into the UI and copy/paste my prompt into the box, it works successfully.

POSITIONAL ONE: 0.75
POSITIONAL ONE: 0.15

Steps to reproduce the problem

Go to the API panel (http://127.0.0.1:7860/docs)
Post to the txt2img endpoint with a prompt that uses loractl syntax

What should have happened?

Applies the loractl

What browsers do you use to access the UI ?

Google Chrome

Sysinfo

sysinfo-2024-11-30-15-29.json

Console logs

venv "C:\0_SD\stable-diffusion-webui-reForge\venv\Scripts\Python.exe"
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Version: f1.1.0-v1.10.1RC-previous-557-g2ec985a4
Commit hash: 2ec985a48398c4639062c35612788284681fcc6b
CUDA 12.1
Launching Web UI with arguments: --pin-shared-memory --cuda-stream --api --listen --api-server-stop
Total VRAM 12282 MB, total RAM 32536 MB
Set vram state to: NORMAL_VRAM
Always pin shared GPU memory
Device: cuda:0 NVIDIA GeForce RTX 4070 Ti : native
Hint: your device supports --cuda-malloc for potential speed improvements.
VAE dtype: torch.bfloat16
CUDA Stream Activated:  True
C:\0_SD\stable-diffusion-webui-reForge\venv\lib\site-packages\transformers\utils\hub.py:127: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(
Using pytorch cross attention
ControlNet preprocessor location: C:\0_SD\stable-diffusion-webui-reForge\models\ControlNetPreprocessor
C:\0_SD\stable-diffusion-webui-reForge\venv\lib\site-packages\diffusers\models\transformers\transformer_2d.py:34: FutureWarning: `Transformer2DModelOutput` is deprecated and will be removed in version 1.0.0. Importing `Transformer2DModelOutput` from `diffusers.models.transformer_2d` is deprecated and this will be removed in a future version. Please use `from diffusers.models.modeling_outputs import Transformer2DModelOutput`, instead.
  deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message)
WARNING:dinov2:xFormers not available
WARNING:dinov2:xFormers not available
09:44:26 - ReActor - STATUS - Running v0.7.1-b2 on Device: CUDA
Loading model sdxl\realcartoonXL_v7.safetensors [d766a0808d] (1 of 1)
Loading weights [d766a0808d] from C:\0_SD\stable-diffusion-webui-reForge\models\Stable-diffusion\sdxl\realcartoonXL_v7.safetensors
2024-11-30 09:44:27,304 - ControlNet - INFO - ControlNet UI callback registered.
model_type EPS
UNet ADM Dimension 2816
Running on local URL:  http://0.0.0.0:7860
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE

To create a public link, set `share=True` in `launch()`.
Startup time: 24.1s (prepare environment: 7.5s, import torch: 3.3s, import gradio: 1.0s, setup paths: 1.2s, initialize shared: 0.2s, other imports: 0.8s, load scripts: 3.8s, create ui: 1.2s, gradio launch: 4.5s, add APIs: 0.6s).
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  11025.796875
[Memory Management] Model Memory (MB) =  2144.3535232543945
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  7857.4433517456055
Moving model(s) has taken 0.67 seconds
Model sdxl\realcartoonXL_v7.safetensors [d766a0808d] loaded in 10.9s (load weights from disk: 1.0s, forge load real models: 9.0s, calculate empty prompt: 0.9s).
NAMES: ['PE_Grimace']
POSITIONAL ONE: 0.75
NAMES: ['PE_Grimace', 'ral-mold-sdxl']
POSITIONAL ONE: 0.15
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\PE_Grimace.safetensors for SDXL-UNet with 722 keys at weight 0.75 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\PE_Grimace.safetensors for SDXL-CLIP with 264 keys at weight 0.75 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\ral-mold-sdxl.safetensors for SDXL-UNet with 722 keys at weight 0.15 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\ral-mold-sdxl.safetensors for SDXL-CLIP with 264 keys at weight 0.15 (skipped 0 keys)
To load target model SDXLClipModel
Begin to load 1 model
Reuse 1 loaded models
[Memory Management] Current Free GPU Memory (MB) =  9211.609375
[Memory Management] Model Memory (MB) =  0.0
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  8187.609375
Moving model(s) has taken 1.08 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  9176.75146484375
[Memory Management] Model Memory (MB) =  4897.086494445801
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3255.664970397949
Moving model(s) has taken 2.99 seconds
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]NAMES: ['PE_Grimace']
POSITIONAL ONE: 0.75
NAMES: ['PE_Grimace', 'ral-mold-sdxl']
POSITIONAL ONE: 0.15
 50%|█████████████████████████████████████████                                         | 15/30 [00:07<00:07,  2.06it/s]NAMES: ['PE_Grimace']|█████████████████████████████████                                 | 15/30 [00:06<00:07,  2.04it/s]
POSITIONAL ONE: 0.38
NAMES: ['PE_Grimace', 'ral-mold-sdxl']
POSITIONAL ONE: 0.3
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\PE_Grimace.safetensors for SDXL-UNet with 722 keys at weight 0.38 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\PE_Grimace.safetensors for SDXL-CLIP with 264 keys at weight 0.38 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\ral-mold-sdxl.safetensors for SDXL-UNet with 722 keys at weight 0.3 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\ral-mold-sdxl.safetensors for SDXL-CLIP with 264 keys at weight 0.3 (skipped 0 keys)
To load target model SDXL
Begin to load 1 model
Reuse 1 loaded models
[Memory Management] Current Free GPU Memory (MB) =  8454.73291015625
[Memory Management] Model Memory (MB) =  4210.9375
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3219.79541015625
Moving model(s) has taken 0.75 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:15<00:00,  1.89it/s]
To load target model AutoencoderKL█████████████████████████████████████████████████████| 30/30 [00:14<00:00,  2.03it/s]
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  6019.41943359375
[Memory Management] Model Memory (MB) =  159.55708122253418
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  4835.862352371216
Moving model(s) has taken 0.50 seconds
Total progress: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:16<00:00,  1.85it/s]
WARNING:root:Sampler Scheduler autocorrection: "DPM++ 2M" -> "DPM++ 2M", "karras" -> "Karras"0 [00:16<00:00,  2.03it/s]
NAMES: ['PE_Grimace']
POSITIONAL ONE: 0.75@0.0,0.38@0.5
activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000001B5958573A0>, <modules.extra_networks.ExtraNetworkParams object at 0x000001B4A6D125F0>]: ValueError
Traceback (most recent call last):
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\extra_networks.py", line 135, in activate
    extra_network.activate(p, extra_network_args)
  File "C:\0_SD\stable-diffusion-webui-reForge\extensions-builtin\Lora\extra_networks_lora.py", line 35, in activate
    te_multiplier = float(params.positional[1]) if len(params.positional) > 1 else 1.0
ValueError: could not convert string to float: '0.75@0.0,0.38@0.5'

To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  10774.5546875
[Memory Management] Model Memory (MB) =  2144.3535232543945
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  7606.2011642456055
Moving model(s) has taken 0.98 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  8935.9609375
[Memory Management] Model Memory (MB) =  4897.086494445801
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3014.874443054199
Moving model(s) has taken 1.08 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00,  3.85it/s]
Memory cleanup has taken 5.97 seconds██████████████████████████████████████████████████| 30/30 [00:07<00:00,  3.85it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:14<00:00,  2.11it/s]
WARNING:root:Sampler Scheduler autocorrection: "DPM++ 2M" -> "DPM++ 2M", "karras" -> "Karras"0 [00:14<00:00,  3.85it/s]
NAMES: ['PE_Grimace']
POSITIONAL ONE: 0.75@0.0,0.38@0.5
activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000001B5947A1DE0>, <modules.extra_networks.ExtraNetworkParams object at 0x000001B595857FD0>]: ValueError
Traceback (most recent call last):
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\extra_networks.py", line 135, in activate
    extra_network.activate(p, extra_network_args)
  File "C:\0_SD\stable-diffusion-webui-reForge\extensions-builtin\Lora\extra_networks_lora.py", line 35, in activate
    te_multiplier = float(params.positional[1]) if len(params.positional) > 1 else 1.0
ValueError: could not convert string to float: '0.75@0.0,0.38@0.5'

To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  9498.73046875
[Memory Management] Model Memory (MB) =  2144.3535232543945
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  6330.3769454956055
Moving model(s) has taken 0.45 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  8960.6357421875
[Memory Management] Model Memory (MB) =  4897.086494445801
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3039.549247741699
Moving model(s) has taken 2.30 seconds
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00,  3.90it/s]
To load target model AutoencoderKL███████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00,  4.14it/s]
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  5925.134765625
[Memory Management] Model Memory (MB) =  159.55708122253418
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  4741.577684402466
Moving model(s) has taken 0.34 seconds
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:08<00:00,  3.64it/s]
WARNING:root:Sampler Scheduler autocorrection: "DPM++ 2M" -> "DPM++ 2M", "karras" -> "Karras"████████████████████████████| 30/30 [00:08<00:00,  4.14it/s]
NAMES: ['PE_Grimace']
POSITIONAL ONE: 0.75@0.0,0.38@0.5
activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000001B59590B6D0>, <modules.extra_networks.ExtraNetworkParams object at 0x000001B50F05AF80>]: ValueError
Traceback (most recent call last):
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\extra_networks.py", line 135, in activate
    extra_network.activate(p, extra_network_args)
  File "C:\0_SD\stable-diffusion-webui-reForge\extensions-builtin\Lora\extra_networks_lora.py", line 35, in activate
    te_multiplier = float(params.positional[1]) if len(params.positional) > 1 else 1.0
ValueError: could not convert string to float: '0.75@0.0,0.38@0.5'

To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  10752.66845703125
[Memory Management] Model Memory (MB) =  2144.3535232543945
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  7584.3149337768555
Moving model(s) has taken 4.03 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  8980.06640625
[Memory Management] Model Memory (MB) =  4897.086494445801
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3058.979911804199
Moving model(s) has taken 1.15 seconds
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00,  4.09it/s]
Memory cleanup has taken 0.86 seconds████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00,  4.13it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:08<00:00,  3.48it/s]
NAMES: ['PE_Grimace']████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:08<00:00,  4.13it/s]
POSITIONAL ONE: 0.75
NAMES: ['PE_Grimace', 'ral-mold-sdxl']
POSITIONAL ONE: 0.15
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\PE_Grimace.safetensors for SDXL-UNet with 722 keys at weight 0.75 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\PE_Grimace.safetensors for SDXL-CLIP with 264 keys at weight 0.75 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\ral-mold-sdxl.safetensors for SDXL-UNet with 722 keys at weight 0.15 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\ral-mold-sdxl.safetensors for SDXL-CLIP with 264 keys at weight 0.15 (skipped 0 keys)
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  5761.109375
[Memory Management] Model Memory (MB) =  2144.3535232543945
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  2592.7558517456055
Moving model(s) has taken 0.51 seconds
To load target model SDXL
Begin to load 1 model
Reuse 1 loaded models
[Memory Management] Current Free GPU Memory (MB) =  3905.59521484375
[Memory Management] Model Memory (MB) =  0.0
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  2881.59521484375
ERROR:root:ERROR lora diffusion_model.output_blocks.2.1.transformer_blocks.8.ff.net.0.proj.weight CUDA out of memory. Tried to allocate 50.00 MiB. GPU 0 has a total capacty of 11.99 GiB of which 0 bytes is free. Of the allocated memory 10.86 GiB is allocated by PyTorch, and 293.92 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
ERROR:root:ERROR lora diffusion_model.output_blocks.2.1.transformer_blocks.8.ff.net.0.proj.weight CUDA out of memory. Tried to allocate 50.00 MiB. GPU 0 has a total capacty of 11.99 GiB of which 0 bytes is free. Of the allocated memory 10.86 GiB is allocated by PyTorch, and 291.56 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Traceback (most recent call last):
  File "C:\0_SD\stable-diffusion-webui-reForge\modules_forge\main_thread.py", line 37, in loop
    task.work()
  File "C:\0_SD\stable-diffusion-webui-reForge\modules_forge\main_thread.py", line 26, in work
    self.result = self.func(*self.args, **self.kwargs)
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\txt2img.py", line 114, in txt2img_function
    processed = processing.process_images(p)
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\processing.py", line 823, in process_images
    res = process_images_inner(p)
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\processing.py", line 981, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\processing.py", line 1356, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\sd_samplers_kdiffusion.py", line 251, in sample
    sampling_prepare(self.model_wrap.inner_model.forge_objects.unet, x=x)
  File "C:\0_SD\stable-diffusion-webui-reForge\modules_forge\forge_sampler.py", line 129, in sampling_prepare
    model_management.load_models_gpu(
  File "C:\0_SD\stable-diffusion-webui-reForge\ldm_patched\modules\model_management.py", line 496, in load_models_gpu
    loaded_model.model_load(async_kept_memory)
  File "C:\0_SD\stable-diffusion-webui-reForge\ldm_patched\modules\model_management.py", line 331, in model_load
    raise e
  File "C:\0_SD\stable-diffusion-webui-reForge\ldm_patched\modules\model_management.py", line 327, in model_load
    self.real_model = self.model.patch_model(device_to=patch_model_to) #TODO: do something with loras and offloading to CPU
  File "C:\0_SD\stable-diffusion-webui-reForge\ldm_patched\modules\model_patcher.py", line 301, in patch_model
    self.patch_weight_to_device(key, device_to)
  File "C:\0_SD\stable-diffusion-webui-reForge\ldm_patched\modules\model_patcher.py", line 279, in patch_weight_to_device
    temp_weight = ldm_patched.modules.model_management.cast_to_device(weight, device_to, torch.float32, copy=True)
  File "C:\0_SD\stable-diffusion-webui-reForge\ldm_patched\modules\model_management.py", line 675, in cast_to_device
    return tensor.to(dtype, copy=copy, non_blocking=non_blocking)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 50.00 MiB. GPU 0 has a total capacty of 11.99 GiB of which 0 bytes is free. Of the allocated memory 10.83 GiB is allocated by PyTorch, and 325.47 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
CUDA out of memory. Tried to allocate 50.00 MiB. GPU 0 has a total capacty of 11.99 GiB of which 0 bytes is free. Of the allocated memory 10.83 GiB is allocated by PyTorch, and 325.47 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
*** Error completing request
*** Arguments: ('task(ujxqtp04jtb10mk)', <gradio.routes.Request object at 0x000001B510C17370>, "Grimace, <lora:PE_Grimace:0.75@0.0,0.38@0.5> from McDonald's, covered in green and black mold, ral-mold, <lora:ral-mold-sdxl:0.15@0.0,0.3@0.5>, with a sad expression, in a dark and damp environment, realistic and detailed illustration", 'low quality, jpeg artifacts, blurry, poorly drawn, ugly, worst quality', [], 1, 1, 7, 1024, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', 'Automatic', '', '', [], 0, 30, 'Restart', 'Automatic', False, '', 0.8, 1545405592, False, -1, 0, 0, 0, True, False, False, 'SDXL', '', 'Equal Weights', 832, 1216, False, 'x264', 'blend', 10, 0, 0, False, True, True, True, 'intermediate', 'animation', None, False, '0', '0', 'inswapper_128.onnx', 'CodeFormer', 1, True, 'None', 1, 1, False, True, 1, 0, 0, False, 0.5, True, False, 'CUDA', False, 0, 'None', '', None, False, False, 0.5, 0, 'tab_single', False, False, 0, None, [], 0, False, [], [], False, 0, 1, False, False, 0, None, [], -2, False, [], False, 0, None, None, ControlNetUnit(input_mode=<InputMode.SIMPLE: 'simple'>, use_preview_as_input=False, batch_image_dir='', batch_mask_dir='', batch_input_gallery=[], batch_mask_gallery=[], multi_inputs_gallery=[], generated_image=None, mask_image=None, hr_option=<HiResFixOption.BOTH: 'Both'>, enabled=False, module='None', model='None', weight=1, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, processor_res=-1, threshold_a=0.5, threshold_b=0.5, guidance_start=0, guidance_end=1, pixel_perfect=True, control_mode=<ControlMode.BALANCED: 'Balanced'>, advanced_weighting=None, save_detected_map=True), ControlNetUnit(input_mode=<InputMode.SIMPLE: 'simple'>, use_preview_as_input=False, batch_image_dir='', batch_mask_dir='', batch_input_gallery=[], batch_mask_gallery=[], multi_inputs_gallery=[], generated_image=None, mask_image=None, hr_option=<HiResFixOption.BOTH: 'Both'>, enabled=False, module='None', model='None', weight=1, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, processor_res=-1, threshold_a=0.5, threshold_b=0.5, guidance_start=0, guidance_end=1, pixel_perfect=True, control_mode=<ControlMode.BALANCED: 'Balanced'>, advanced_weighting=None, save_detected_map=True), ControlNetUnit(input_mode=<InputMode.SIMPLE: 'simple'>, use_preview_as_input=False, batch_image_dir='', batch_mask_dir='', batch_input_gallery=[], batch_mask_gallery=[], multi_inputs_gallery=[], generated_image=None, mask_image=None, hr_option=<HiResFixOption.BOTH: 'Both'>, enabled=False, module='None', model='None', weight=1, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, processor_res=-1, threshold_a=0.5, threshold_b=0.5, guidance_start=0, guidance_end=1, pixel_perfect=True, control_mode=<ControlMode.BALANCED: 'Balanced'>, advanced_weighting=None, save_detected_map=True), False, False, 7, 1, 'Constant', 0, 'Constant', 0, 1, 'enable', 'MEAN', 'AD', 1, False, 1.01, 1.02, 0.99, 0.95, False, 0.5, 2, False, 256, 2, 0, False, False, 3, 2, 0, 0.35, True, 'bicubic', 'bicubic', False, 0.5, 0.18, 15, 1, False, 5.42, 0.28, False, 0.7, False, 'Discrete', 'v_prediction', True, 'v_prediction', 120, 0.002, False, 'MultiDiffusion', 768, 768, 64, 4, False, False, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, 0, False, False, False, False, False, False, 0, False, 'Positive', 0, ', ', 'Generate and always save', 32, '') {}
    Traceback (most recent call last):
      File "C:\0_SD\stable-diffusion-webui-reForge\modules\call_queue.py", line 74, in f
        res = list(func(*args, **kwargs))
    TypeError: 'NoneType' object is not iterable

---
NAMES: ['PE_Grimace']
POSITIONAL ONE: 0.75
NAMES: ['PE_Grimace', 'ral-mold-sdxl']
POSITIONAL ONE: 0.15
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  8896.02587890625
[Memory Management] Model Memory (MB) =  4897.086494445801
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  2974.939384460449
Moving model(s) has taken 1.04 seconds
  0%|                                                                                                                                                                                                                           | 0/30 [00:00<?, ?it/s]NAMES: ['PE_Grimace']
POSITIONAL ONE: 0.75
NAMES: ['PE_Grimace', 'ral-mold-sdxl']
POSITIONAL ONE: 0.15
 50%|█████████████████████████████████████████████████████████████████████████████████████████████████████████                                                                                                         | 15/30 [00:07<00:07,  2.07it/s]NAMES: ['PE_Grimace']|█████████████████████████████████████████████████████████████████████████████████████████████████                                                                                                 | 15/30 [00:06<00:07,  2.06it/s]
POSITIONAL ONE: 0.38
NAMES: ['PE_Grimace', 'ral-mold-sdxl']
POSITIONAL ONE: 0.3
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\PE_Grimace.safetensors for SDXL-UNet with 722 keys at weight 0.38 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\PE_Grimace.safetensors for SDXL-CLIP with 264 keys at weight 0.38 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\ral-mold-sdxl.safetensors for SDXL-UNet with 722 keys at weight 0.3 (skipped 0 keys)
[LORA] Loaded C:\0_SD\stable-diffusion-webui-reForge\models\Lora\sdxl\ral-mold-sdxl.safetensors for SDXL-CLIP with 264 keys at weight 0.3 (skipped 0 keys)
To load target model SDXL
Begin to load 1 model
Reuse 1 loaded models
[Memory Management] Current Free GPU Memory (MB) =  8204.02294921875
[Memory Management] Model Memory (MB) =  4210.9375
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  2969.08544921875
Moving model(s) has taken 0.74 seconds
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:15<00:00,  1.99it/s]
Memory cleanup has taken 0.18 seconds██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:14<00:00,  2.04it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:15<00:00,  1.88it/s]
WARNING:root:Sampler Scheduler autocorrection: "Euler" -> "Euler", "None" -> "Automatic"████████████████████████████████████████████████████████████████████████████████████████████
NAMES: ['PE_Grimace']██████████████| 30/30 [00:15<00:00,  2.04it/s]
POSITIONAL ONE: 0.75@0.0,0.38@0.5
activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000001B50F0D23B0>, <modules.extra_networks.ExtraNetworkParams object at 0x000001B50F0D30A0>]: ValueError
Traceback (most recent call last):
  File "C:\0_SD\stable-diffusion-webui-reForge\modules\extra_networks.py", line 135, in activate
    extra_network.activate(p, extra_network_args)
  File "C:\0_SD\stable-diffusion-webui-reForge\extensions-builtin\Lora\extra_networks_lora.py", line 35, in activate
    te_multiplier = float(params.positional[1]) if len(params.positional) > 1 else 1.0
ValueError: could not convert string to float: '0.75@0.0,0.38@0.5'

To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  5821.22802734375
[Memory Management] Model Memory (MB) =  2144.3535232543945
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  2652.8745040893555
Moving model(s) has taken 0.63 seconds
To load target model SDXL
Begin to load 1 model
Reuse 1 loaded models
[Memory Management] Current Free GPU Memory (MB) =  8284.216796875
[Memory Management] Model Memory (MB) =  4210.9375
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3049.279296875
Moving model(s) has taken 1.13 seconds
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:04<00:00, 11.56it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:04<00:00, 11.85it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:04<00:00, 12.11it/s]

Additional information

No response

1rre commented 2 days ago

What's happening looks to be that the loractl extra network isn't activated, I think you need to do this from somewhere in the request but I'm not certain, it could be alwayson_scripts, or something else... Can you post the details of a generation done from the UI (under the generated photo with seed, model, dimensions etc.), as it may be necessary to specify that the extra network should be activated there. It may alternatively/additonally be necessary to add loractl to alwayson_scripts in your request json.

1rre commented 2 days ago

There could be a bug here with 'NoneType' object is not iterable but I think that's an edge case where I don't check if model copy failed (which it looks to here due to the out of memory error) - if it does then I imagine we want to throw an error anyway rather than continuing with the previous model

I'll also look into potential memory optimisations as I suspect that I don't release the GPU memory until after copy or even later (end of generation?) but I'm currently unsure how to even monitor that

altoiddealer commented 2 days ago

There could be a bug here with 'NoneType' object is not iterable but I think that's an edge case where I don't check if model copy failed (which it looks to here due to the out of memory error) - if it does then I imagine we want to throw an error anyway rather than continuing with the previous model

I'll also look into potential memory optimisations as I suspect that I don't release the GPU memory until after copy or even later (end of generation?) but I'm currently unsure how to even monitor that

The OOM was a separate issue for sure - I’ll see about the script activation… I wouldn’t think that’s the issue though, because the script is enabled by default when I launch ReForge

altoiddealer commented 2 days ago

What's happening looks to be that the loractl extra network isn't activated, I think you need to do this from somewhere in the request but I'm not certain, it could be alwayson_scripts, or something else... Can you post the details of a generation done from the UI (under the generated photo with seed, model, dimensions etc.), as it may be necessary to specify that the extra network should be activated there. It may alternatively/additonally be necessary to add loractl to alwayson_scripts in your request json.

You are CORRECT!

It seems that, despite the extension being enabled by default in my UI settings, the script is still disabled by default via API.

I added the following to my discord bot, which enables the script via the API call:

self.img_payload['alwayson_scripts'].setdefault('dynamic lora weights (reforge)', {}).setdefault('args', []).append({'Enable Dynamic Lora Weights': True})

Panchovix commented 2 days ago

That does happen because, the setting to be enabled or disabled on the UI is set the value you have on ui-config.json

While, the extension itself is disabled by default on the code.

Nice catch though! I have it disabled by default for combability and such.

altoiddealer commented 1 day ago

@Panchovix btw, I have a very advanced discord bot that, a very long time ago, I implemented this idea I had which is to automatically calculate and apply loractl scaling values, when there are multiple LORAs triggered by the bot.

The way I have this written it modifies the values before they get inserted into the prompt... looking at this now, seems like something I should be doing after they get inserted.

Nonetheless, I'm mentioning this in case this sparks any light bulbs in your own head - such as adding similar functionality to the built-in extension

https://github.com/altoiddealer/ad_discordbot/blob/main/bot.py#L2500-L2543

https://github.com/altoiddealer/ad_discordbot/blob/main/settings_templates/config.yaml#L91-L100

Panchovix / stable-diffusion-webui-reForge