glibsonoran / Plush-for-ComfyUI

Custom node for ComfyUI/Stable Diffustion
GNU General Public License v3.0
142 stars 11 forks source link

Seed for OAI Dall-E node #42

Closed alessandroperilli closed 7 months ago

alessandroperilli commented 7 months ago

Per our conversation on Reddit, thanks again for making the changes to this node. It now works great.

One issue I still have is that, if the prompt doesn't change, the node refuses to generate a new image.

Sometimes, this is a good thing: I may want to tweak the settings of an inpainting node that manipulates the generated Dall-E 3 image, and I don't want a new generation every time.

Other times, I really need Dall-E 3 to generate alternative versions of the same prompt and it doesn't seem to be a way to do that other than kill ComfyUI and restart it.

Is it possible to add a seed to the node, or another mechanism that forces the regeneration of the image when the user wants it?

Thank you!

glibsonoran commented 7 months ago

Yes, I'm sure I can fix that for you. That's a great catch as currently the node doesn't work in the flow like you'd expect a KSampler to.

Also coming out later today , the Dall-e3 node will allow batches of up to 8 images.

glibsonoran commented 7 months ago

OK, here's what I've done for a first pass: = I added a 'seed' input that's set to forceInput=True, meaning it only appears as an input connection not a value in the UI. = You can attach a "seed" generator node to this input like: 'Seed' from the ttN suite, or "Random Number" from the WAS Node Suite. Attach the 'seed' or the 'int' output to my 'seed' input and set the generator node to "Randomize" or "Increment" if you want the node to respond to each Queue, or set it to 'fixed' if you only want it to respond once. (I'm sure you know this its the same behavior as the KSampler seed.

This should work for now and it's quick. In future I may create an input in the node's UI that allows the user to select: 'Run Always'/'Run Once', since the OpenAI API doesn't accept or expose any seeds and calling it a seed may be misleading. Changing it has no effect on a latent or the image. It's merely a device to reset a value in the node automatically (or not) so it will run with each queue (or not).

I'll give you a head's up when I upload my next revision that will include this.

glibsonoran commented 7 months ago

OK, slight revision I made the seed value part of the UI with its own built-in selection for "fixed", "randomize" etc. So you can use it as a UI element or convert it to an input if you want. Basically its just like the seed control in a KSampler.

The files for 1.20.3 are up now and in addition to the new mock 'seed', Dall-e can do image batches, up to 8 images. So let me know if you have any issues.

If you get an opportunity take a look at my style_prompt node and let me know if you see anything that could be improved.

Thanks hope your next workflow goes well

alessandroperilli commented 7 months ago

@glibsonoran Thank you for these modifications!

The approach works well, except for one detail:

I started with a Fixed seed to verify that the default behaviour of the node was unaltered. Check. Then, I changed to Randomize and hit Queue, expecting the new generation to be different. It wasn't.

The node effectively created a new Random image, but only after an extra Fixed image.

I generated several Random images, and then I changed the value back to Fixed, to keep the last seed I was using. The node ignored me. It did one extra Random image, and only then, it decided to pay attention to the Fixed value.

glibsonoran commented 7 months ago

Thanks for the update, let me see if I can address your concerns:

You've probably already read this but just to reiterate: Seed values mean nothing to Dall-e, the Dall-e API has no ability to pass seed values. And Dall-e generates its own internal prompt (based on your prompt) on each run, so no two runs with the same external prompt (your prompt) will produce the same image. I'm just using seed as a device to control when the node runs.


OK, I set up the Dall-e node and a standard KSampler in the same workflow and changed back and forth from fixed to randomize seed. They both worked exactly the same way which makes sense because the implementation for the behavior of a seed input sets itself up automatically, it uses an internal Comfy process, I don't code it. So what I saw is that both nodes acted this way:

= set to fixed: It makes one image using the current seed value and then won't make any more and the seed remains unchanged after the run.

=change to randomize from fixed: It refuses to make another image but the seed changes at the end of the Queue. (Note this is because 'control_after_generate' is the only available setup. This means Comfy sees the same seed when you press Queue so no image is generated. It's not until after the Queue ends that the two nodes make a new random seed.) After this run that produced no image, Queuing again generates an image using the random seed that was generated at the end of the 'no image' run.

=Leaving it on Randomize: It generates a new image for each Queue.

=Changing back to fixed: Because it generated a random seed at the end of the prior run (when it was set to randomize), it runs another image based on the new random seed value it created. However at the end of this Queue no new seed is generated. Further Queuing now results in no image.

Both the standard KSampler and the Dall-e node exhibit this behavior.


Now if I use Inspire pack's "Global Seed", it has a 'control_before_Generate' setting , which I'll use for this description, and it overrides the seed settings on all nodes that have a seed input without requiring a noodle connection.

=set to fixed: Both node's seed values are coerced to the seed value in Global Seed and an image is generated on that value. Subsequent attempts to generate an image produce nothing as expected.

=change to randomize from fixed: This time Queueing results in an image on the first try as Global Seed creates the new random seed before image generation begins.

=Left on randomize: It generates a new image for each Queue

=Changing back to fixed. Because Global Seed generates its seed before generation begins, it didn't generate a new seed at the end of the last Queue. Now since it's set to fixed it doesn't generate a seed at the beginning of the Queue either. So immediately no new image is generated.

This is why I use Global Seed in my workflows, its behavior is more immediate without the artifact of one extra image after changing to fixed or needing two Queues to start generation on a node changed from fixed to randomize.
But in my testing my node behaves exactly like a KSampler.

Anyway , I hope this helps. Let me know if you are still having issues. Thanks.

alessandroperilli commented 7 months ago

You are right. In the AP Workflow, I use the SD Parameter Generator, which includes the Seed Generator code from rgthree, which behaves like the Global Seed in the Inspire Pack. I used the SD Parameter Generator for so many months that I completely forgot how the default KSampler and seed generation work.

Sorry for the unnecessary trouble.