search

kohya-ss / sd-webui-additional-networks

GNU Affero General Public License v3.0
1.74k stars 294 forks source link

[Bug]: Sending To I2I causing doubling up of Lora Weight #172

Open rushuna86 opened 1 year ago

rushuna86 commented 1 year ago

Whenever you generate and image using Additional Networks instead of built in prompt lora and send to Image2Image, a doubling up of lora weight happens. From the console, I can see a network is generated for the lora weight settings, image is generated. When a new image is generated the following "restoring last networks | 0/25 [00:00<?, ?it/s] original forward/weights is restored." Happens before new network is created. However it seems like this process isn't being done when image is sent to Image2Image from Text2Image. If you send image from PNG info no double up happens, only from text2image

So the image is sent to Image2Image/Inpaint/Sketch/etc generated network weight, then the Additional Networks will also be enabled with the same weight you set once again. Double up the weight and overfitting. https://imgur.com/a/3pk8MAJ

  1. Generated Grid with Lora weight 0.8 picked top left picture to send to inpaint.
  2. Drew over hand inside inpaint, left all setting as is. Inpaint is drawn at 0.8 weight x 2 ( this also happens in image2image).
  3. Once this doubling of weight happens you go back into regular Text2Image and generate again the images are Lora weight x 2 Inside Image2image if you scroll down and disable "Additional Networks" it will generate with Lora Weight x 1, but if you do this text2image even if "Additional Networks" is enabled it will act like it's disabled. Another fix I found is when this Lora doubling of weight happens in image2image if you go back to text2image and disable Additional Networks and generate an image. Image2Image will generate with Lora weight x 1.
rushuna86 commented 1 year ago

I also found that if you send from text2image to image2image, and scroll down to Additional Networks and change the lora weight value it will also trigger "restoring last networks | 0/25 [00:00<?, ?it/s] original forward/weights is restored." before generation inside Image2image, otherwise this doesn't occur and Additional Networks in image2image will stacked lora weight on top of the network created in text2image.

rushuna86 commented 1 year ago

seems like commit 383abc0 also fixed this issue

rushuna86 commented 1 year ago

my mistake it still does the doubling up, easily replicated when sending from t2i to inpainting. going down to additional networks and changing any value of fixes the doubling effect