asagi4 / comfyui-prompt-control

ComfyUI nodes for prompt editing and LoRA control
GNU General Public License v3.0
191 stars 16 forks source link

clip skip with sdxl throwing error #32

Closed alenknight closed 5 months ago

alenknight commented 7 months ago

when i run an SDXL model, with clip skip (as in with efficient loader, or with "clip set last layer"... i get this error

image

asagi4 commented 7 months ago

This is a bug with ComfyUI_ADV_CLIP_emb nodes. My code just uses it. I have a local patch to it like this:

diff --git a/adv_encode.py b/adv_encode.py
index 9e0cbb1..9f6d4d0 100644
--- a/adv_encode.py
+++ b/adv_encode.py
@@ -223,7 +223,7 @@ def encode_token_weights_l(model, token_weight_pairs):

 def encode_token_weights(model, token_weight_pairs, encode_func):
     if model.layer_idx is not None:
-        model.cond_stage_model.clip_layer(model.layer_idx)
+        model.cond_stage_model.set_clip_options({"layer": model.layer_idx})

     model_management.load_model_gpu(model.patcher)
     return encode_func(model.cond_stage_model, token_weight_pairs)

but the problem should be reported against the other repo. I just fixed it for myself and forgot about it.

alenknight commented 7 months ago

ah ok, i'll put it on the comfyui repo... can you help us understand the bug so we can report it to comfyanonymous?

also, how did you patch it? where did you put that code to resolve the bug?

asagi4 commented 7 months ago

You misunderstood. It's not a Comfy bug, it's a bug with BlenderNeko's ComfyUI_ADV_CLIP_emb extension nodes. The patch goes into that repository.

My nodes have an integration that uses code from those nodes if you have them installed.

asagi4 commented 7 months ago

Looks like there's already a pull request to fix things: https://github.com/BlenderNeko/ComfyUI_ADV_CLIP_emb/pull/21

asagi4 commented 7 months ago

If you don't want to fix the issue yourself you can work around the problem by removing the ADV_CLIP_emb nodes until they're fixed.

alenknight commented 7 months ago

interesting... so when they patch then i can update and get it and use theirs. but this was happening with Efficiency nodes too... pretty much any node that called clip skip.
still all depending on their code base?

as for how to fix... i saw your code... but where do i inject that if i want to fix it in the meantime?

asagi4 commented 7 months ago

Notice how the error and patch both mention adv_encode.py? That's the file that you need to edit. The patch tells you what you need to modify (- is removal, + is add, the rest is context)

asagi4 commented 5 months ago

The problem is fixed in the advanced encoding extension, so I'll close this