search

asagi4 / comfyui-prompt-control

ComfyUI nodes for prompt editing and LoRA control
GNU General Public License v3.0
207 stars 16 forks source link

Recommendations for composite SD1.5/SDXL workflow #27

Closed ghostsquad closed 9 months ago

ghostsquad commented 10 months ago

I have a SD1.5 workflow that seems to work really well right now. I'd like to be able to use SDXL as well and seamlessly switch between the models without having to do much work. I've been told that using the same prompt for CLIP_L and CLIP_G is perfectly fine and still works pretty well. I was wondering if you have any recommendations or examples for a workflow that works well for both SD1.5 and SDXL without manual modifications when switching to an SDXL model?

I was considering doing some text manipulation, like looking at the checkpoint name, and looking for sdxl or sd_xl in the name, and based on that, taking the prompt, duplicating it, with and wrapping with CLIP_L( .. ) CLIP_G( .. ). But before I go through all that effort, I was wondering if you have any recommendations.

asagi4 commented 10 months ago

You don't really need the SDXL or CLIP_L functions to use SDXL; Many models do indeed work just fine with just the basic prompt. I don't really use them myself because I haven't figured out how they're useful; I tend to get better results without them. They're just there in case someone needs them.

But if you want to, you could use tag-filtered prompts, and just filter out SDXL-related things when you're not using an SDXL model, for example

prompt goes here [sdxl stuff goes here CLIP_L(foo) SDXL(1,2,3,4):XL]

And then add a FilterSchedule with the tag filter set to XL when an SDXL model is in use.

Also, there's no CLIP_G function. The prompt that's not in CLIP_L is the CLIP_G prompt, for both SDXL and SD1.5. And in the case where you don't explicitly specify a CLIP_L prompt, the CLIP_G prompt is used for it (this is ComfyUI's default behaviour)

ghostsquad commented 10 months ago

What about the SDXL(1,2,3,4) part. There are other nodes that make management of the width and height vars much easier than having it in the prompt. Along those lines, I was also considering prompt manipulation to construct and inject a string, but I feel like it would be better not to have to do that.

asagi4 commented 10 months ago

I don't really have anything specific for that. For the vast majority of my SDXL gens I don't set those parameters at all.

ghostsquad commented 10 months ago

Oh, are those parameters needed if using a different aspect ratio than 1024x1024?

asagi4 commented 10 months ago

No, you can gen using other aspect ratios and it will still work. Though SDXL has specific resolutions you should stick to for best results.

The parameters might help or hinder generations, I honestly have no idea; I haven't experimented enough with them to say what effects they actually have. It probably depends heavily on the model you're using, too.

ghostsquad commented 10 months ago

Ok. I guess I need to play more with these things and come back with a concrete ask once I have some more information/experience/feedback.

asagi4 commented 9 months ago

I added some nodes that will help with integrating nodes that calculate SDXL parameters. I'm going to close this issue for now. See the README for details.