AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
139.45k stars 26.44k forks source link

[Bug]: Order of Operations is Not Documented #13790

Open Meerkov opened 10 months ago

Meerkov commented 10 months ago

Is there an existing issue for this?

What happened?

I've been experimenting with prompt weighting/blending, and the results are inconsistent with the documentation.

Example:

Crystal containing elemental [fire:1.0|ice:1.0]
Crystal containing elemental {fire:1.0|ice:1.0}
Crystal containing elemental (fire:1.0|ice:1.0)
Crystal containing elemental [fire|ice]:1.0
Crystal containing elemental {fire|ice}:1.0
Crystal containing elemental (fire|ice):1.0

Every single one of these produces a unique image.

This seems to imply that the documentation is not correct when it discussed how weighting and prompt blending works. https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#attentionemphasis The cheat sheet implies that weighting by 1.0 should not effect the prompt.

Note that this is not using any LORA which is mentioned as being incompatible with this syntax. This is just base SDXL, and yet still the prompt weighting and blending is inconsistent.

I assume I'm misunderstanding something, so can I ask for a more detailed FAQ answer on how to use this syntax to explain where the misconception is?

Steps to reproduce the problem

Try to make a prompt blended image with equal weights using different brackets. Notice that the bracket type effects the resul.

What should have happened?

The FAQ implies that the bracket type does not matter when a weight is provided explicitly.

Sysinfo

Windows 10

What browsers do you use to access the UI ?

No response

Console logs

N/A

Additional information

No response

Meerkov commented 10 months ago

I did more tests to try to narrow down the issue in https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/13791

This ticket should focus on the documentation of the order of operations.

I believe one of the issues here is the following:

| operator happens first, before weights are applied.

This causes [fire:1.0|ice:1.0] to alternate between A [fire:1.0:1.0] and B [ice:1.0]

It's not clear at all what A does. I think this changes it from a "weight" to a prompt schedule syntax, resulting in unexpected behavior.