receyuki / comfyui-prompt-reader-node

The ultimate solution for managing image metadata and multi-tool compatibility. ComfyUI node version of the SD Prompt Reader
MIT License
290 stars 22 forks source link

[BUG] - Model output impairs LoRA application #27

Closed alessandroperilli closed 11 months ago

alessandroperilli commented 11 months ago

Description

First: it took me countless hours over an entire week to pinpoint this behavior. Yet, I am not sure this is a bug. It might be that I don't understand well enough the intricacies of ComfyUI. With that in mind, here's the problem:

When the Model output of an SD Parameter Generator node is passed to an Efficient KSampler node, whatever LoRA has been defined in a Efficient Loader node gets ignored.

Reproduction steps

Recreate the following situation:

Screenshot 2023-12-03 at 09 12 14

As you can see in this example, the Cartoon_SDXL_V2 LoRA is completely ignored by the Efficient KSampler node if I use the Model output generated by the SD Parameter Generator node.

However, if I use the Model output coming straight from an Efficient Loader node, the LoRA is correctly applied:

Screenshot 2023-12-03 at 09 13 27

In both cases, notice this is an img2img workflow simply VAE Encoding a source image.

If this is not a bug, but the expected behaviour, it makes impossible for me to use this portion of the SD Parameter Generator node in the AP Workflow, so I really hope it's a bug.

Thank you!

Image file

Output when the Model comes from SD Parameter Generator:

Efficient_LoRA_Test_00007_

Output when the Model comes from Efficient Loader:

Efficient_LoRA_Test_00008_

receyuki commented 11 months ago
Screenshot 2023-12-03 at 09 12 14

I've set up a new workflow for testing, and everything is working fine, including Lora. However, when I tried to understand your workflow, I noticed that you forgot to apply Lora to the model. You need to apply Lora to both the model and the clip for it to work properly

截屏2023-12-07 上午4 12 08
alessandroperilli commented 11 months ago

I realized this morning, but didn't have a second to close the issue. Sorry about that.

Efficiency nodes are amazing, but sometimes they make you forget how many things they do behind the scene and if you change something, you make these silly mistakes. Sorry again.