cheald / sd-webui-loractl

An Automatic1111 extension for dynamically controlling the weights of LoRAs during image generation
MIT License
239 stars 10 forks source link

Text encoder, unet control don't working? #24

Open MetroByte opened 11 months ago

MetroByte commented 11 months ago

<lora:name:te=0@0,1@1:unet=1> (the same as mentioned in readme.md with pixelart robot) gives the same result as <lora:name:te=0:unet=1>

Unet progression seems to be working as intended. However, simple progression like <lora:name:0@0,1@1> works fine.

I can't figure out what is the problem. I have extension for LoRA's (LoRA Block Weight) but that is disabled.

Latest version of SD-webui and this extension (at the moment of issue creating)

cheald commented 11 months ago

I'm unable to replicate this. Separate te/unet control seems to be working as expected here:

<lora:pixel:te=0@0,1@1:unet=1@0,0@1>

image

<lora:pixel:te=0@0,1@1:unet=1>

image

The "simple progression" is just setting that schedule on both the te and unet together:

image

When you capture plots of the weights (with the checkbox in the extension options), do you get a plot that reflects what you expect? It's possible that you're working with a Lora that's heavily dependent on the text encoder, so starting at 0 weight is basically starting it outside of "lora space" and it has trouble getting back into it. Perhaps try starting the TE at some value larger than 0.

MetroByte commented 11 months ago

Plotting works fine. I've even added prints to network_patch.py functions to check if calculation works. And everything works as intended, I guess. At least calculation. And once again, I've done example setting right from readme text, last example (only with different LoRA). It is no difference what to put after , in my case. It seems like it is taking the first te number and keeps it? Maybe another extension conflicts with this one xyz_grid-0002-123456

cheald commented 11 months ago

I think there might be something funny happening with the webui itself. After a totally fresh boot of the app, I get:

image

But then when I run a second generation without changing anything:

image

That's unexpected, to say the least. It might be that some weights might not be getting unwound properly, or something. I'll dig around a bit.

MetroByte commented 11 months ago

; semicolon?

cheald commented 11 months ago

Commas and semicolons are both valid delimiters. I just use them because I always forget to quote the individual chunks in the S/R matrix.

MetroByte commented 11 months ago

Okay, I got it. I disabled another extensions and re-launched SD-webui. No changes even after fresh re-launch, same results.

Unet progression works. xyz_grid-0007-123456

another edit: Actually, after another fresh restart, te progression is working. And just once. Then stops working. Really odd. Also, after this, images generated with te=1:unet=1@0,0@2 are equal to 1@0,0@2.