kijai / ComfyUI-Florence2

Inference Microsoft Florence2 VLM
MIT License
692 stars 47 forks source link

[Feature Request] Add a seed field to avoid running the whole flow with fixed seed. #36

Open GalaxyTimeMachine opened 3 months ago

GalaxyTimeMachine commented 3 months ago

I have a large workflow that starts with the Florence2 nodes, and sometimes I want to re-use a seed to see small changes I make to the flow's settings. Because Florence2 runs every time, it causes the whole workflow to run, instead of the just the changed parts. Would adding a seed field allow that part to be "fixed", so it doesn't repeatedly run when the seed doesn't change?

webspam0 commented 3 months ago

I saw this issue title and was curious. The issue you're describing is not how ComfyUI works. Either input to the Florence-2 node is changing or some other code is causing it. I recommend you try isolation testing.

GalaxyTimeMachine commented 3 months ago

Normally, if a seed is fixed and nothing in the ComfyUI workflow changes, the workflow should have nothing to do, so will not run again. If you make a small change on a node, then everything from that node onwards will run, but nothing before it should, because it hasn't changed. However, the Florence2 node will create a different output every time you run the workflow, so it cause the whole workflow to run, even if it has fixed seeds on everything, and nothing else has changed. I often use a "Global Seed" node, so everything can be fixed in place from that single node, but Florence2 makes it useless. The only workaround, I've found, is to cut and paste the FL2 prompt into a text box and bypass the FL2 nodes to use that text box for the prompt. Surely there has to be a less cumbersome method!? These images have a fixed seed in the KSampler (the only place to set it here), yet it will keep creating different images, because the Florence2 node has no "seed", or similar, to tie it to a single output from the same image. Screenshot 2024-07-18 115016 Screenshot 2024-07-18 115106

GalaxyTimeMachine commented 3 months ago

Strangely, I'm running it again and I consistently get the same as the second image, and it is cached, the way I'd expect it to be. Ah! It must be switching between workflow that causes my problem. It means the image can not be recreated from the same seed, because the LLM may interpret the image differently on other runs....which is why I wanted to know if it's possible to add a seed type of option to the node and if that would make images re-creatable.

webspam0 commented 3 months ago

Right. That makes a lot more sense. If you restart ComfyUI, it will re-run all nodes. Changing the entire graph (which happens when loading a workflow), deleting / recreating input nodes... etc.

Turn do_sample off, ensure num_beams and max tokens match the original generation.

Bonus workarounds: Read exif from a previously generated image. Pretty sure by default the first prompt Comfy finds is added - no need to parse the entire workflow. Or use save/load text nodes using seed filename.