FizzleDorf / ComfyUI_FizzNodes

Custom Nodes for Comfyui
MIT License
386 stars 60 forks source link

Batch Prompt Schedule don't work #58

Closed FourTest closed 6 months ago

FourTest commented 9 months ago

Hi FizzleDorf, Batch Prompt Schedule don't work. print_out : got prompt

Max Frames: 61 frame index: 0 Current Prompt: High detail, a girl, yellow dress Next Prompt: High detail, a girl, red dress Strength : 1.0

Max Frames: 61 frame index: 1 Current Prompt: High detail, a girl, yellow dress Next Prompt: High detail, a girl, red dress Strength : 0.9166666666666666 ...... Max Frames: 61 frame index: 11 Current Prompt: High detail, a girl, yellow dress Next Prompt: High detail, a girl, red dress Strength : 0.08333333333333337

Max Frames: 61 frame index: 12 Current Prompt: High detail, a girl, red dress Next Prompt: High detail, a girl, red dress Strength : 1.0

Max Frames: 61 frame index: 13 Current Prompt: High detail, a girl, red dress Next Prompt: High detail, a girl, red dress Strength : 0.9795918367346939

Max Frames: 61 frame index: 14 Current Prompt: High detail, a girl, red dress Next Prompt: High detail, a girl, red dress Strength : 0.9591836734693877

my prompt 17046875728436

the result 17046877562837

no yellow dress girl.all are red dress.

tobiaswuerth commented 9 months ago

I agree, the default behaviour is confusing. One would assume that it automatically interpolates smoothly from one prompt to the other. But actually it just decreases one prompt until its strength reaches 0 and then uses the next prompt. See this example I made:

"0": "AAAAAAAAAA",
"4": "BBBBBBBBBB",
"8":"CCCCCCCCCC",

prints

 Max Frames:  8
 frame index:  0
 Current Prompt:  pre  AAAAAAAAAA  app
 Next Prompt:  pre  BBBBBBBBBB  app
 Strength :  1.0

 Max Frames:  8
 frame index:  1
 Current Prompt:  pre  AAAAAAAAAA  app
 Next Prompt:  pre  BBBBBBBBBB  app
 Strength :  0.75

 Max Frames:  8
 frame index:  2
 Current Prompt:  pre  AAAAAAAAAA  app
 Next Prompt:  pre  BBBBBBBBBB  app
 Strength :  0.5

 Max Frames:  8
 frame index:  3
 Current Prompt:  pre  AAAAAAAAAA  app
 Next Prompt:  pre  BBBBBBBBBB  app
 Strength :  0.25

 Max Frames:  8
 frame index:  4
 Current Prompt:  pre  BBBBBBBBBB  app
 Next Prompt:  pre  CCCCCCCCCC  app
 Strength :  1.0

 Max Frames:  8
 frame index:  5
 Current Prompt:  pre  BBBBBBBBBB  app
 Next Prompt:  pre  CCCCCCCCCC  app
 Strength :  0.75

 Max Frames:  8
 frame index:  6
 Current Prompt:  pre  BBBBBBBBBB  app
 Next Prompt:  pre  CCCCCCCCCC  app
 Strength :  0.5

 Max Frames:  8
 frame index:  7
 Current Prompt:  pre  BBBBBBBBBB  app
 Next Prompt:  pre  CCCCCCCCCC  app
 Strength :  0.25

but what I actually want is this:

 Max Frames:  8
 frame index:  0
 Current Prompt:  pre  AAAAAAAAAA  app
 Next Prompt:  pre  (AAAAAAAAAA: 0.75), (BBBBBBBBBB: 0.25)  app
 Strength :  1.0

 Max Frames:  8
 frame index:  1
 Current Prompt:  pre  (AAAAAAAAAA: 0.75), (BBBBBBBBBB: 0.25)  app
 Next Prompt:  pre  (AAAAAAAAAA: 0.5), (BBBBBBBBBB: 0.5)  app
 Strength :  1.0

 Max Frames:  8
 frame index:  2
 Current Prompt:  pre  (AAAAAAAAAA: 0.5), (BBBBBBBBBB: 0.5)  app
 Next Prompt:  pre  (AAAAAAAAAA: 0.25), (BBBBBBBBBB: 0.75)  app
 Strength :  1.0

 Max Frames:  8
 frame index:  3
 Current Prompt:  pre  (AAAAAAAAAA: 0.25), (BBBBBBBBBB: 0.75)  app
 Next Prompt:  pre  BBBBBBBBBB  app
 Strength :  1.0

 Max Frames:  8
 frame index:  4
 Current Prompt:  pre  BBBBBBBBBB  app
 Next Prompt:  pre  (BBBBBBBBBB: 0.75), (CCCCCCCCCC: 0.25)  app
 Strength :  1.0

 Max Frames:  8
 frame index:  5
 Current Prompt:  pre  (BBBBBBBBBB: 0.75), (CCCCCCCCCC: 0.25)  app
 Next Prompt:  pre  (BBBBBBBBBB: 0.5), (CCCCCCCCCC: 0.5)  app
 Strength :  1.0

 Max Frames:  8
 frame index:  6
 Current Prompt:  pre  (BBBBBBBBBB: 0.5), (CCCCCCCCCC: 0.5)  app
 Next Prompt:  pre  (BBBBBBBBBB: 0.25), (CCCCCCCCCC: 0.75)  app
 Strength :  1.0

 Max Frames:  8
 frame index:  7
 Current Prompt:  pre  (BBBBBBBBBB: 0.25), (CCCCCCCCCC: 0.75)  app
 Next Prompt:  pre  CCCCCCCCCC  app
 Strength :  1.0

Which I only managed to achieve when configuring each frame manually like:

"0": "AAAAAAAAAA",
"1": "(AAAAAAAAAA: `0.75`), (BBBBBBBBBB: `0.25`)",
"2": "(AAAAAAAAAA: `0.5`), (BBBBBBBBBB: `0.5`)",
"3": "(AAAAAAAAAA: `0.25`), (BBBBBBBBBB: `0.75`)",
"4": "BBBBBBBBBB",
"5": "(BBBBBBBBBB: `0.75`), (CCCCCCCCCC: `0.25`)",
"6": "(BBBBBBBBBB: `0.5`), (CCCCCCCCCC: `0.5`)",
"7": "(BBBBBBBBBB: `0.25`), (CCCCCCCCCC: `0.75`)",
"8":"CCCCCCCCCC",

Is this intended behaviour? Can you provide an example what the easiest way is to achieve such smooth transitions using your framework? Thanks

tobiaswuerth commented 9 months ago

meanwhile I've create a utility function for myself to generate the prompt schedule like I want it to:

def create_smooth_transition_schedule(prompts, key_frame_interval):
    import numpy as np
    interval = np.linspace(0, 1, key_frame_interval+1)[1:-1]
    interval_reversed = interval[::-1]

    schedule = []
    for i in range(len(prompts)-1):
        p1 = prompts[i]
        schedule.append(p1)

        p2 = prompts[i+1]
        for w1, w2 in zip(interval_reversed, interval):
            s = f'({p1}: `{w1:0.3f}`), ({p2}: `{w2:0.3f}`)'
            schedule.append(s)

    schedule.append(prompts[-1])

    for i in range(len(schedule)):
        print(f'"{i}": "{schedule[i]}",')

create_smooth_transition_schedule([
    "AAAAAAAAAA",
    "BBBBBBBBBB",
    "CCCCCCCCCC",
], key_frame_interval=4)

prints

"0": "AAAAAAAAAA",
"1": "(AAAAAAAAAA: `0.750`), (BBBBBBBBBB: `0.250`)",
"2": "(AAAAAAAAAA: `0.500`), (BBBBBBBBBB: `0.500`)",
"3": "(AAAAAAAAAA: `0.250`), (BBBBBBBBBB: `0.750`)",
"4": "BBBBBBBBBB",
"5": "(BBBBBBBBBB: `0.750`), (CCCCCCCCCC: `0.250`)",
"6": "(BBBBBBBBBB: `0.500`), (CCCCCCCCCC: `0.500`)",
"7": "(BBBBBBBBBB: `0.250`), (CCCCCCCCCC: `0.750`)",
"8": "CCCCCCCCCC",

.. for your case this prints:

"0": "yellow dress",
"1": "(yellow dress: `0.917`), (red dress: `0.083`)",
"2": "(yellow dress: `0.833`), (red dress: `0.167`)",
"3": "(yellow dress: `0.750`), (red dress: `0.250`)",
"4": "(yellow dress: `0.667`), (red dress: `0.333`)",
"5": "(yellow dress: `0.583`), (red dress: `0.417`)",
"6": "(yellow dress: `0.500`), (red dress: `0.500`)",
"7": "(yellow dress: `0.417`), (red dress: `0.583`)",
"8": "(yellow dress: `0.333`), (red dress: `0.667`)",
"9": "(yellow dress: `0.250`), (red dress: `0.750`)",
"10": "(yellow dress: `0.167`), (red dress: `0.833`)",
"11": "(yellow dress: `0.083`), (red dress: `0.917`)",
"12": "red dress",

is this what you want?

FourTest commented 9 months ago

Thank u tobiaswuerth.

I don't want to use transition effects So I rewrote the prompt word,as: "0": "AAAAAAAAAA", "24": "AAAAAAAAAA", "25": "BBBBBBBBBB", "36": "BBBBBBBBBB"

So, I only need to discard 2 frames (24 and 25)

FizzleDorf commented 9 months ago

I agree, the default behaviour is confusing. One would assume that it automatically interpolates smoothly from one prompt to the other. But actually it just decreases one prompt until its strength reaches 0 and then uses the next prompt. ... Is this intended behaviour? Can you provide an example what the easiest way is to achieve such smooth transitions using your framework? Thanks

this is because the schedule is using composable diffusion, not changing the individual prompt weights. This is automating the Conditioning (Average) node to string many interpolations in a row. If you want to do the weights separately, you can use outside tools like Parseq to interpolate the weights per frame. this is also using the value schedule and the prompt weight variables and the prompt schedule. I'll make sure I make note of this in the wiki/README update.

FizzleDorf commented 9 months ago

I've fixed the issue with the value schedule in the latest commit. I'll be uploading example workflows today. I'll see about adding the smooth transition code to the repo as well in alternate nodes. Thanks for the info.

FizzleDorf commented 8 months ago

64 fixed this issue

Herambnaik commented 6 months ago

@FizzleDorf unable to use prompt scheduler could you please check? image

FizzleDorf commented 6 months ago

hey @Herambnaik, I'll look into it. thanks for reporting the issue.

FizzleDorf commented 6 months ago

this seems to be a conflict with the naming conventions of this node https://github.com/Big-Idea-Technology/ComfyUI-Book-Tools