ModelSurge / sd-webui-comfyui

An extension to integrate ComfyUI workflows into the Webui's pipeline
MIT License
504 stars 32 forks source link

:) AMAZING extention, i just have a question #175

Closed 4lt3r3go closed 10 months ago

4lt3r3go commented 10 months ago

why there are some stuff that are appyed ot top? like face restoreation or Adetailer for example. I wish there were a way to use Comfy nodes as last step in the chain, the post/pre selector doesnt help in this case here

PladsElsker commented 10 months ago

Hello, thank you for using the extension! I'm glad you like it.

Regarding your question, I am not sure I fully understand.

If you want to use ComfyUI nodes to do some post-processing on your image, you can do so by developing ComfyUI accordions in the txt2img and img2img tabs. You can then enable the workflow types that you want, and add some nodes between the From Webui and To Webui nodes.

Is this what you are trying to do?

PladsElsker commented 10 months ago

Oh I think I understand. You're right, you cannot have a workflow type as the last step of the chain right now. Only "somewhat close to" the last step of the pipeline is possible with the postprocess workflow type.

The reason for this is because we are kind of restricted on when we can run the workflow types in the Webui. I wonder how easily it could be done right now.

In the meanwhile, you can always run the pipeline once without ComfyUI, and then pass it to img2img with 0 denoising strength to do your postprocess in the tab. I know it's not very practical like this, but at least it works.

PladsElsker commented 10 months ago

So I just looked, and there is one script callback that could be used for this, postprocess_image.

I think it's somewhat easy to add a workflow type there, but we need to add restrictions on how many To Webui nodes can be added (only one in this case), and throw an error otherwise.

I think it's very doable if we add this restriction feature to the extension's workflow type API.

4lt3r3go commented 10 months ago

thanks for answering... i think i understand why is not possible (probably is the latent to image step). Yeah in my case i was adding some sharpeness and grain using comfy nodes, but then after the process the adetailers or face restoration kicks in and remove the sharpness and the grain, processing over it. I think you got the "problem" yes...

4lt3r3go commented 10 months ago

so after those comfy nodes the image comes out from the latent space and being processed again AS IMAGE by those face modules, i think is like that i dont know

PladsElsker commented 10 months ago

Yes, basically the face-restore section is right after the extension's callback. So it's just a matter of running the extension at the right moment.

I actually don't know if we really need to restrict at all the amount of To Webui nodes. I'll look into it this week.

PladsElsker commented 10 months ago

I can confirm, this is needed:

restrictions on how many To Webui nodes can be added (only one in this case), and throw an error otherwise.

4lt3r3go commented 10 months ago

forgive me brother, havent really understood what is that "to webui" restriction. It is something i can do? sorry again

PladsElsker commented 10 months ago

forgive me brother, havent really understood what is that "to webui" restriction. It is something i can do? sorry again

You don't have to do anything about it. I was documenting what needed to change in the issue dedicated to the problem. If you are interested in what's going on under the hood, this info could be of interest to you. Otherwise, you can ignore my previous comment 👍

PladsElsker commented 10 months ago

I made a PR for this that basically adds 2 new workflows for processing later in the pipeline:

There are specificities to each workflow that might be useful to know. I will add this later to the wiki when the PR is merged.

Postprocess workflow: ✔️ Can modify the size of the batch ❌ Is not the very last thing that runs in the pipeline (but almost)

Postprocess image workflow: ❌ Cannot modify the size of the batch (only 1 To Webui node is allowed to exist at all times in the workflow) ✔️ Runs later in the pipeline (notably after face restoration)

Before save image workflow: ❌ Cannot modify the size of the batch (only 1 To Webui node is allowed to exist at all times in the workflow) ❌ Does not update the metadata of the image ❌ Does not update the image in the viewport/preview ✔️ Runs even later in the pipeline (notably after color correction)

4lt3r3go commented 10 months ago

GOD BLESS YOU! (does god exists? i dont think so...whatever)

4lt3r3go commented 10 months ago

WORKS PERFECTLY!! thank you!!! the only problem i still find (this was already happening) is that often when i paste an image the comfy settings are not loaded at all. i see them in the exif, they are pasted in the prompt but arent loaded. this wasnt happening the first time i installed this extention... i think i may have broke something. anyway. i just need 2 nodes, they are simple to setup. I can do it manually. no worries

4lt3r3go commented 10 months ago

this error in console when i press SAVE image. (images are saved tho) 231119124451

PladsElsker commented 10 months ago

Ah, that's an issue I introduced with the PR indeed. I'll fix this today.

Thanks for reporting!

PladsElsker commented 10 months ago

the only problem i still find (this was already happening) is that often when i paste an image the comfy settings are not loaded at all.

What do you mean by "comfy settings"? Do you mean that the workflow is not being loaded when drag and dropping an image? Or something else?

4lt3r3go commented 10 months ago

yes. workflow is not loaded anymore. It may be me, no worries. When i started using this extention and saved some images, then when i was dropping saved images into prompt or PNG info tab to recall all saved settings, the comfyu extention was getting all settings too. This is not happening anymore for some reasons. the extention stays empy and dont get all the settings i used previously. Again, not a problem. You already helped me so much. I'm just using 2 nodes in the end.. they are easy to rebuild.

This is amazing. Thanks again! :D

PladsElsker commented 10 months ago

Oh I think you're running into #112.

4lt3r3go commented 9 months ago

Oh I think you're running into #112.

i'm becoming autistic (thank you for help) is there a way to setup a workflow as default so is always automatically loaded? this is my obsession btw and what is all this post about :D

sharp_grain.json

PladsElsker commented 9 months ago

I think #113 would help with that. If you think overriding the default workflow would still be usefull after #113 is implemented, feel free to open an issue for a feature request!

PladsElsker commented 9 months ago

FYI right now the default workflows are hardcoded in js, so there is no way at the moment.