opparco / stable-diffusion-webui-two-shot

Latent Couple extension (two shot diffusion port)
MIT License
715 stars 151 forks source link

Bad results when work with LoRA #29

Open QR-0W opened 1 year ago

QR-0W commented 1 year ago

The first picture is the result when no LoRA is used. 图片 The second picture is the result when work with some LoRA. 图片 I tried to change the weight of LoRA, but failed to get a satisfying result.

Padusarbor commented 1 year ago

Did you use one lora twice? https://github.com/opparco/stable-diffusion-webui-composable-lora/issues/7 It seems okay when I use different loras but when I duplicate them, the image becomes corrupted.

basedholychad commented 1 year ago

i think the guy that make these 2 extensions probably dead , the composable lora extension never seem to work for me

CamiloMM commented 1 year ago

i think the guy that make these 2 extensions probably dead , the composable lora extension never seem to work for me

To be fair, things are moving so fast (DALL-E was the stone ages!) that he's either long dead or went to the bathroom for longer than usual.

Padusarbor commented 1 year ago

If you used the rectangular mode, and then switched to the sketch mode, or if you reloaded webui, try updating the sketch and the prompt with the according buttons.

basedholychad commented 1 year ago

i think i figure out ''a bit''. i've retrained 2 lora to test with high resolution (1024x1024) and with no noise offset and low epoch, both lora same settings and it's seem work better than 2 lora with different config/resolution so i think either 1 of your lora might be overcook/diffrent train resolution with another that may cause this

QR-0W commented 1 year ago

i think i figure out ''a bit''. i've retrained 2 lora to test with high resolution (1024x1024) and with no noise offset and low epoch, both lora same settings and it's seem work better than 2 lora with different config/resolution so i think either 1 of your lora might be overcook/diffrent train resolution with another that may cause this

Yes. After some tests, I find that some 'hard' LORAs may cause this problem when use this extension. Even if it worked well when running separately, it corrupts the image when composed together.

prescience-data commented 1 year ago

I've had some success with the following:

  1. Disable Composable LoRA if you have it enabled.

  2. Using this tool, slice your grid with specific focus on the elements you're targeting with a particular LoRA.

  3. Describe each segment, but activate each LoRA only once in the first place you use it. Additionally, when using these tighter slices, I found it was important to redefine some of the basic background descriptions in each layer to avoid the strength differential (ie 0.2 vs 0.8) creating halos.

  4. Continue to use tokens from a previously activated LoRA in subsequent prompts.

  5. Create a matching stack of negative prompts (all identical to start), and then use ( ) to dial down the keywords from the secondary LoRAs (or keywords embedded within the same LoRA). Importantly, if you have slices that overlap, make sure to use [ ] to soften the blow as overlapping ( ) modifiers can get nasty.

image

andupotorac commented 1 year ago
  1. Composable

But if you disable Composable Lora, won't that break the Prompt structure? basically what will be the point of using AND anymore?