TinyTerra / ComfyUI_tinyterraNodes

A selection of nodes for Stable Diffusion ComfyUI
GNU General Public License v3.0
393 stars 45 forks source link

unexpected prompt behavior with xyPlot and list of inputs #127

Open rjgoif opened 3 months ago

rjgoif commented 3 months ago

Take the following workflow (which uses a custom node that is one of many that will iterate over a list to add to the queue)

The prompt output box suggests that the prompt is correctly changing per iteration from the loader, but the Ksampler only "sees" the first prompt.

I tried using pipe and non-pipe versions, no difference. I updated my nodes, also no difference. I also tried some similar "string list" nodes as inputs, still no effect.

Any ideas as to how to fix the problem? Some of my iterated prompt lists are huge and I can't input them any other way without manually (or with an OS level script) inputting them. Problem with this is that comfy starts to have input lag after like 50-60 prompts, and the internals don't get updated fast enough leading to shuffled prompt orders or duplicated/dropped prompts. It's a mess.

unexpected prompt decoupling

Note I edited the prompt list before submitting the image. That's why there are three output prompts but only two inputs. Ignore that.

TinyTerra commented 3 months ago

Interesting... I'll have to have a look into how the custom node works behind the scenes. If its purely queueing a new prompt with an enumerated text output, I'd expect the plot to output separate xy's for each positive (ie 3 lots of two images for the cfg change) not a group of 6... If it 'queues' within the same execution then i can understand why its messing up.

But in any case something seems weird there.

What is your expected outcome? or what are you trying to achieve more specifically? I'm assuming 3 sets of 2 image plots?

I don't have time right now, but I'll have a closer look and work on a fix tomorrow.

rjgoif commented 3 months ago

What is your expected outcome? or what are you trying to achieve more specifically? I'm assuming 3 sets of 2 image plots?

In my example I didn't hook up an xyplot preview, but yes. The first prompt in the prompt list should have fed in triggering two output images, then the second prompt triggering two new images with prompt 2. The really strange thing is that the loader node is correctly outputting a list of two prompt strings. But the strings that are leaving the loader node are not those that are used as inputs in the sampler node.

If I hook up a preview to the xyplot image output, I will get two grids, both with the first prompt.

Of note, if I don't use the xyplot option, everything works as expected: the sampler receives a list of two inputs and creates a list of two correct outputs.

I'd upload more examples, but I am at work and can't comfy from here (sadly).

rjgoif commented 3 months ago

Here we have an example working as expected. A list of prompts goes in, and a list of outputs matching the prompts comes out.

unexpected prompt decoupling 02

Next, another variation showing the bug. I even process the list of text prompts separately with their own clip encoder after verifying that the loader node took them as inputs correctly. Still doesn't work. unexpected prompt decoupling 03

It appears that the bug is specific to how the xy node works, not the loader/sampler nodes.

TinyTerra commented 3 months ago

Okay, I see what the issue is. The xyplot wasn't designed with list inputs in mind for the actual plotting side. (the execution uses core comfy elements which is why the correct 4 images are being output individually in temp).

This might be a little more involved than i initially thought, but i'll work on it.

rjgoif commented 3 months ago

I guess another option would be to add a Z axis which was a variable that could have a new XY plot under each value. That's what other XY plotters do (e.g. A1111, xyAny)

From there, I guess you could make an arbitrary number of dimensions, too. an XY plot for combo of values from Z1, Z2, Z3, etc.

TinyTerra commented 3 months ago

Sorry i haven't had as much time to work on this.

I have it "working" with lists to a point, except now it doubles up on the outputs. I sort of expected the comfy list support to be within the base sampling functions - but somehow instead it re-executes the entire node. as of yet i haven't had much luck back tracing how that works in the base comfy code.

If i get the list support working properly, i probably wont add the Z axis - adding it properly with the list input as well will get too complex... But maybe on that note it would make more sense to use Z-axis instead of supporting lists..

TinyTerra commented 3 months ago

Okay, adding the Z-Axis was easier than tracing through comfys code.. I'll also add a advPlot String node (Similar to the range one) for node based string plotting instead of having to parse your longer lists into the xyPlot syntax.

Just want to test it a bit further before pushing

rjgoif commented 3 months ago

Does it output a series of XY images, one for each Z value? Or does it concatenate it all into one giant superimage?

TinyTerra commented 3 months ago

a series of XY plots

image

TinyTerra commented 3 months ago

I've pushed the changes if you want to update and have a play with it.

rjgoif commented 3 months ago

Hey this works really well! I want to try some creative ways to expand Z into combinatorial iterations, but I am crunched at work for the moment.

But great work getting the implementation up and running! I think anyone scared of using Comfy vs A1111 can now get basically full functionality from just your pack alone without having to learn the "scary" noodle constructions.