ljleb / prompt-fusion-extension

auto1111 webui extension for all sorts of prompt interpolations!
MIT License
259 stars 16 forks source link

[(token:1)::] assertion error #50

Closed freezerain closed 1 year ago

freezerain commented 1 year ago

[(token:1)::] format causing exception intended behaviour should be ignore this formats means effect of the token will be from the begining to the end in my case its not possible to use [(token:1):0.0:1.0] as bypass because Im using dynamic prompt extension to build prompt: [(token){:|::}{|0.2}] meaning it can start or finish on some T (0.2 in this case) and using 0.0 and 1.0 will meant disabling effect in 50% of cases

I hope it make sense, have a nice day💪

ljleb commented 1 year ago

I'm not sure I understand why you want to omit the step number after ::. What is that supposed to mean? It looks like it does not have a really useful meaning by itself. The webui does not allow to skip the step numbers normally, and I think it is a bad idea to change existing semantics.

IIUC, I believe [(token){:|::}{-1|0.2}] or {[(token){:|::}0.2]|(token)|} should do the job as a workaround for now. if token is very long, dynamic prompts has prompt variables that you can use to deduplicate the repeating words.

ljleb commented 1 year ago

Now, prompt editing with omitted step numbers should be forwarded to the webui parser without raising an error.

freezerain commented 1 year ago

Thank. I will explain. First of all with fussion disabled I get (almost 90% sure that I am) expected behaviour of [token::] which is from 0.0 to 1.0, So if I understand correctly webUI is allow to skip the step number, and in their github wiki there are atleast 1 example with first number skipped [token::0.5]

Now about why, by using string like [__animal__{:|::}{|0.2}] I can get 3 state:

  1. [__animal__:] or [__animal__::] which is full effect
  2. [__animal__::0.2] only first 20%
  3. [__animal__:0.2] only last 80%

I have few prompts person-generator that have multiple core variables that I looking for and with this type of prompting I can leave machine for night render/seed discovery session and get very variative results Example: {|[({{{rim|volumetric|cinematic} lighting}|crepuscular rays|backlight|dimly}:{0.6|1}){:|::}{|0.2}], } p.s. Yes, I code java/c#, so naturally I got boner on constructions like {{{([[()]])}}}

ljleb commented 1 year ago

I think I understand your use case, however I am certain your assumption that [::] is a valid builtin webui construct is wrong. I verified this by inspecting what the webui parser returns when [::] is used as a text prompt literally: it turns out the webui forwards [::] literally to the CLIP text encoder. As soon as I put a number, i.e. [::3], then instead, 2 empty prompts are passed to the CLIP text encoder (technically just 1 empty prompt since the webui caches duplicate prompt embeddings when encoded simultaneously).

You can actually verify this by generating [::], then pin the seed, then generate another image with [::5] for example. If you change the step number from [::5] to [::12] it should give the same result, but it should be different from just [::].

But putting that aside. Even with the updated code, do you get different results depending on whether the extension is disabled or not? Or I didn't get that right...? As the extension now forwards to the webui the invalid construct [a:b:], it should use the behavior of the webui, so there really should not be any difference.

If there is still an issue, then if you could share a prompt that unexpectedly gives different results depending on whether the extension is enabled or not, I could reproduce the problem and add test cases to fix.

freezerain commented 1 year ago

Lets test in isolation, same seed. I think its clear that except [token:0.0:1.0] and slight variation with [token::] (step offset -+1, rounding error or prompt length change) everything else works as I assumed. My guess its a dynamic prompt extension or maybe buggy test on your side?

a ball [made from the red spikes::] as expected, from 0.0 to 1.0 image a ball [made from the red spikes:0:20] as expected spiky as hell (20 steps, but I think there are 1step offset somwhere) image a ball [made from the red spikes:0.0:1.0] NOT as expected, a bug, my assymtion 1.0 casted to 1 which is 1 step image a ball [made from the red spikes:0.0:0.999] as expected, 0.999 solved the bug image a ball [made from the red spikes::0.1] as expected, not first 10% (increasing number increasing spikeness) image a ball [made from the red spikes:0.1] as expected, only first 10% (increasing value smoothens surface) image

freezerain commented 1 year ago

Oh sry, I forget to add, this is another seed, yes there are difference between renders, but its there even with your extension disabled, there are too much variables why I have different image, but your extension is not(not the only one) source of it. And I dont care honestly. Everything else works fine, bug is solved for me, thank you

01800-1563159047 01801-1563159047

ljleb commented 1 year ago

and slight variation with [token::]

The reason is that the webui will also pass the tokens [ and : and ] to the text encoder in this case. It won't if you specify a step number, for example [token::0.999] or [token::12]. This is because prompt editing without a step number is invalid in the webui syntax. It only generates slight variations because the text encoder has learned to not attribute much importance to these tokens [, : and ] when used in this way to enclose other more meaningful tokens.

So the variations you are seeing are not caused by any extension, it simply is invalid syntax the webui rejects at the moment.