RuntimeError: output with shape [128, 320, 1, 1]

[alexgilseg]

After the last upgrade I get this error when trying to use my newly produced lora in SD:

activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000001F6C7D0C9D0>, <modules.extra_networks.ExtraNetworkParams object at 0x000001F6C7D0C2E0>]: RuntimeError Traceback (most recent call last): File "D:\SD\stable-diffusion-webui\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "D:\SD\stable-diffusion-webui\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "D:\SD\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 214, in load_loras lora = load_lora(name, lora_on_disk.filename) File "D:\SD\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 176, in loadlora module.weight.copy(weight) RuntimeError: output with shape [128, 320, 1, 1] doesn't match the broadcast shape [128, 320, 3, 3]

When producing the lors they make fine sample images that seem to be working well, but it's when adding the lora to SD this error appears. Eveything is up to date. I have added additional networks extension.

[kohya-ss]

The LoRA with Conv2d-3x3 extension (with -network_args "conv_dim=4") don't seem to be supported in bulit-in LoRA in Web UI. Please use additional-networks extension: https://github.com/kohya-ss/sd-webui-additional-networks

[TheOnlyHolyMoly]

@vladmandic I just wanted to draw your attention to this topic as it seems to persist in the built-in lora module SD.Next and a1111. Not even sure who the owner of the built-in lora feature would be. Would you like me to try to reach out to a1111 (e.g. @kamnxt) to address it upstream?

edit: I just recognized the error is coming up when using in combination with a 2.1 Model, but not with a 1.5 Model Base...

23:01:11-443888 ERROR loading Lora D:\VLADautomatic\models\Lora\E-Drive\90sTMNTRaphael_v10Alpha.safetensors: RuntimeError ┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐ │ D:\VLADautomatic/extensions-builtin/Lora\lora.py:220 in load_loras │ │ │ │ 219 │ │ │ │ try: │ │ > 220 │ │ │ │ │ lora = load_lora(name, lora_on_disk.filename) │ │ 221 │ │ │ │ except Exception as e: │ │ │ │ D:\VLADautomatic/extensions-builtin/Lora\lora.py:181 in load_lora │ │ │ │ 180 │ │ with torch.nograd(): │ │ > 181 │ │ │ module.weight.copy(weight) │ │ 182 │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ RuntimeError: output with shape [16, 320] doesn't match the broadcast shape [16, 320, 16, 320]

[vladmandic]

Give me couple of days...

[kamnxt]

...why am I being mentioned here?

[vladmandic]

@TheOnlyHolyMoly the owner of extensions-builtin/Lora is A111 himself - he took Loras code from a long time ago and made it built-in instead of making additional-networks a mandatory extension.

Since then Loras got a lot more new features and most of them are not compatible with old built-in implementation. So people starting using a1111-sd-webui-locon extension which patches <lora:name> namespace with additional functionality.

But patching existing namespace is problematic, so that extension is now EOL and replaced by a1111-sd-webui-lycoris which works perfectly, but you need to use <lyco:name> in your prompts as it registers a new separate namespace instead of patching existing one. Note that this works for both old-loras/new-loras/locons/lycos.

And then you have a sd-webui-additional-networks which is a heavy-weight approach as it has requires usage of dedicated UI to select/enable loras and doesn't rely on prompt parsing as such.

So what's best? IMO built-in Lora for its simplicity + Lyco for everything else. But that causes confusion since users don't know about that. Solution? maybe remove built-in Lora completely and ask lyco author to patch a1111-sd-webui-lycoris to handle both namespaces if built-in lora provider is not found? I don't know.

Suggestions?

[TheOnlyHolyMoly]

@vladmandic great overview, thank you. So for now I can at least help myself using lyco instead.

Looking at the bigger picture: You know my thought is that a built-in extension should be useful for dealing with the standard cases. If it fails on handling newer loras and I need a second extension to handle them, then why not remove the outdated extension from the code base completely? And I know the technology is dynamically expanding at a high rate, so I understand things can date-out easily.

I really like the idea of patching a1111-sd-webui-lycoris to handle all namespaces irrespective of how they are called. If the owner of that repo would be up and willing you'd have a one-fits-all solution and less complexity across different installations of a1111 (ideally just 1 extension instead of now 3). And maybe also remove the necessity to have different folders for lora and lycoris (I am always guessing after downloading a bunch what it might be).

Not sure if I can tag @KohakuBlueleaf from here.

[vladmandic]

And I know the technology is dynamically expanding at a high rate, so I understand things can date-out easily.

Yup. that's exactly why in my repo I chose to link extensions-builtin as git submodules instead of copying the code or doing a one-time clone on install.

[KohakuBlueleaf]

@TheOnlyHolyMoly the owner of extensions-builtin/Lora is A111 himself - he took Loras code from a long time ago and made it built-in instead of making additional-networks a mandatory extension.

Since then Loras got a lot more new features and most of them are not compatible with old built-in implementation. So people starting using a1111-sd-webui-locon extension which patches <lora:name> namespace with additional functionality.

But patching existing namespace is problematic, so that extension is now EOL and replaced by a1111-sd-webui-lycoris which works perfectly, but you need to use <lyco:name> in your prompts as it registers a new separate namespace instead of patching existing one. Note that this works for both old-loras/new-loras/locons/lycos.

And then you have a sd-webui-additional-networks which is a heavy-weight approach as it has requires usage of dedicated UI to select/enable loras and doesn't rely on prompt parsing as such.

So what's best? IMO built-in Lora for its simplicity + Lyco for everything else. But that causes confusion since users don't know about that. Solution? maybe remove built-in Lora completely and ask lyco author to patch a1111-sd-webui-lycoris to handle both namespaces if built-in lora provider is not found? I don't know.

Suggestions?

I can make an option to let extension know if it can patch lora Auto detect is not very a good idea (I have done it in locon ext and it cause lot of bugs for different environment) Add an explicit option is more easy and controlable

[vladmandic]

@KohakuBlueleaf i'd love that! would your code also look-up lora's inside --lora-dir (right now it only looks at --lyco-dir) if that explict option was enabled?

[KohakuBlueleaf]

I can make it look for --lora-dir if enable --lyco-patch-lora

[KohakuBlueleaf]

But, my patch will be like: Make an ExtraNetwork instance which has name='lora' but use lyco backend so and will exist at the same time and will use --lora-dir, lyco will use --lyco-dir

If you want, I can let it to disable when --lyco-patch-lora

[vladmandic]

yup, that sounds good. <lora> will use --lora-dir and <lyco> will use --lyco-dir even if they are using same extranetwork implementation in the backend. no need to do more than that.