search

kijai / ComfyUI-SUPIR

SUPIR upscaling wrapper for ComfyUI
Other
1.59k stars 88 forks source link

how can I apply a lora to sdxl model #75

Closed szRush closed 8 months ago

szRush commented 8 months ago

Thank you for updating since sdxl model is filled in a panel， not a separate checkpoint loading node

kijai commented 8 months ago

Not currently possible directly, as the underlaying SUPIR pipeline doesn't support it. I wanted to look into adding the feature though, for now only way to do that is to save the checkpoint with the LoRA in Comfy first, it's clumsy but it works: image

Latentnaut commented 8 months ago

it works and the outcome is better ;)

kijai commented 8 months ago

it works and the outcome is better ;)

Can I ask you which lora did you use? I still need to test this myself, and find a way to apply it directly if it's really worth it.

Latentnaut commented 8 months ago

it was a Lora I made of my face, not public.

I noticed a slight improvement of the sharpness.

kijai commented 8 months ago

it was a Lora I made of my face, not public.

I noticed a slight improvement of the sharpness.

Ok, understood, thank you for the info!

Also I did figure this out and you can now load it with these optional inputs: image

When they are used the sdxl_model -selection in the model loader is simply ignored, I don't want to remove it yet as it would break old workflows.

HWiese1980 commented 6 months ago

Have there been changes around this? At least I see no error loading a lora into the SDXL model before giving it to SUPIR Model Loader... or how does it show that it doesn't work?

kijai commented 6 months ago

Have there been changes around this? At least I see no error loading a lora into the SDXL model before giving it to SUPIR Model Loader... or how does it show that it doesn't work?

It has worked ever since I added the v2 loader node.

HWiese1980 commented 6 months ago

Cool, I hadn't seen that anywhere. Although I have not noticed any difference in the result between using a lora and not using it.

kijai commented 6 months ago

Cool, I hadn't seen that anywhere. Although I have not noticed any difference in the result between using a lora and not using it.

It does work, but generally you have to give it a bit more freedom to affect the image, as with full or high control strength SUPIR will try it's best to keep faithful to the original image. So that would be higher cfg and I find that lower control scale end value gives it enough freedom while still keeping most of the original.

Can't speak for all LoRAs of course, but I tried couple and they had an effect at least.