search

comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
https://www.comfy.org/
GNU General Public License v3.0
53.72k stars 5.7k forks source link

Bug: CLIP node #3344

Open Agnyy opened 5 months ago

Agnyy commented 5 months ago

CLIP does not work, when I insert a model into a node, a black screen appears.

image image

Is there a test CLIP model? I don’t know what exactly the problem is, either in the model or in the processing. But I really need to solve this. Therefore, I would be very grateful for your help.

ltdrdata commented 5 months ago

image

You can extract clip model through CLIPSave node.

silveroxides commented 5 months ago

Is there a test CLIP model? I don’t know what exactly the problem is, either in the model or in the processing. But I really need to solve this. Therefore, I would be very grateful for your help.

I assume your clip model is one exported using a tool like stable-diffusion-webui-model-toolkit extension. If this is the case the issue with loading is that the extension exports clip without the 'text_model.' prefix for the keys. comfyUI expects the prefix because majority of ComfyUI loading is written around the specs from huggingface and instead of keeping the possibility open to load different formats and properly correct model keys for loading, it just blatantly denies instead. You need to rename the keys using the key list specifications in the components directory in model-toolkit. I have s fork of the extension among my repositories that has fixed support for sdxl and added LoRA support. Safe the clip as . safetensors in the extension then ask chatgpt for a script for loading model and renaming keys follows by saving. Or you can import it into any model using toolkit and use above mentioned option to export it un ComfyUI