[Feature]: Native Weight Blocks

[Holyniwa]

Feature description

I still can't believe that weight blocks isn't a part of SDN... For some reason, their extension (https://github.com/hako-mikan/sd-webui-lora-block-weight) hasn't been working for the last 2 months, since is why I haven't bothered updating, but with LCMs becoming more popular, i really want to get on the newest version.

Before the "it is, your just doing it wrong", no it isn't. I keep seeing messages like ValueError: could not convert string to float: '2,2.5,2.5,2.5,2.5,2.5,2.5,0.5,0.5,0.5,0.5,0.5,0.8,0.8,0.8,0.8,0.8' on the first Lora/Lycoris it hits, and then refuses to load any lora/lycoris following the error.

Version Platform Description

No response

[vladmandic]

It is, but in dev branch and will be merged to master in the next release. Feel free to checkout dev branch if you want to try it out.

[Holyniwa]

was this implemented? not sure how to update to a dev branch. (i am very github noob)

[vladmandic]

It's in the last release, don't need dev branch.

[Holyniwa]

Oh? I updated it to latest release, but it's still not working? Do you have some documentation of your weight blocks or are they supposed to work as they did with the extension? (which I've now disabled) I'm getting the error messages:

│ D:\stable-diffusion\automaticNEW\modules\extra_networks.py:75 in activate │ │ │ │ 74 │ │ try: │ │ > 75 │ │ │ extra_network.activate(p, extra_network_args) │ │ 76 │ │ except Exception as e: │ │ │ │ D:\stable-diffusion\automaticNEW\extensions-builtin\Lora\extra_networks_lora.py:38 in activate │ │ │ │ 37 │ │ │ te_multiplier = float(params.named.get("te", te_multiplier)) │ │ > 38 │ │ │ unet_multiplier = [float(params.positional[2]) if len(params.positional) > 2 │ │ 39 │ │ │ unet_multiplier = [float(params.named.get("unet", unet_multiplier[0]))] * 3 │ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ValueError: could not convert string to float: '0,0.1,0.1,0.1,0.1,0.1,0.1,1,1,1,1,1,1,1,1,1,1'

[vladmandic]

Full docs need to be written, but you'll see a short example in changelog.

[Holyniwa]

Ahh, i see it now. So rather than <lora:epiNoiseoffset_v2:1.75:0,0.1,0.1,0.1,0.1,0.1,0.1,1,1,1,1,1,1,1,1,1,1>

now its

Hmm, i can see how it's easier for newer people to use...a shame to lose control of the individual blocks, but this is still better than nothing. Much appreciate!

[vladmandic]

We may add individual blocks if there is demand.

[Holyniwa]

Right. Dang, I really do hope so 🙏 . I noticed that due to in/mid/out, base is missing as the first block, and 9-11 function extensively different from 12-17. (also, I'm not really an expert on it either, i just had months of testing with them. They're sorta complex, which is why i understand why the community doesn't use them to great details) i also forgot to ask, does this work for Lycoris?

Update: I'm experimenting with it a bit more, and it looks like it may actually have individual weight block support after all. I think it's structured as:

Not sure for Lycoris yet. Will continue experimenting. Update2: Hmm... actually I'm not sure. It's accepting the format, but it's having some odd interactions. Base isn't have as much as an impact as it should. This is it all very weird... Update2.5: omg i just realized i stumbled on the post from on [Apr 12, 2023](https://github.com/vladmandic/automatic/discussions/99). God, I'm too tired for this. No wonder nothing seems to be working as intended. aight. sleep for me.

[vladmandic]

looping in @AI-Casanova as he authored the recent block weight implementation.

[AI-Casanova]

What the others call BASE is 99% just text encoder layers (plus 2 other hard to classify keys)

TE LoRA (if those keys exist) can be accessed by splitting the numerical inputs like this. would be 0.5 TE 1.0 UNet. Then you can add the in=0.75 etc after

[Holyniwa]

Thanks Vlad!

Hey Casanova, first off, thank you so much for adding this feature! It's easily my most used feature in the older versions, and I've been dying to see it implemented (or supported) in the newer updates.

Interesting information regarding BASE. A lot of the information involving block weights still goes a bit over my head, but I appreciate the knowledge!

Regarding this implementation's usage, I tried looking up and down the changelogs, and honestly, I just couldn't find a detailed example.

• Could you help provide a detailed example of how the ins, mid, out are structured for SDN to understand it? as you can see, i tried a number of methods above, and nothing seemed to really click. (that as i was stupidly looking at an old post that provided no useful information)
• Btw, are Lycoris supported? do they use the same method or are they different? (When using the extension I would have it like this: but i get the feeling that won't work, just like the LoRA format wont work).

I know some people used INS, IND, INALL, MIDD, OUTD, OUTS, OUTALL in the past with a different extension, not sure if those are relevant for the structure. (i personally liked having the weight blocks in the prompt rather than in a separate tab) Honestly, anything really helps, lol. Again, thank you!