search

city96 / ComfyUI-GGUF

GGUF Quantization support for native ComfyUI models
Apache License 2.0
966 stars 60 forks source link

Request for chatglm3 model loader or converter for GGUF. #81

Closed doogyhatts closed 2 months ago

doogyhatts commented 2 months ago

The GGUF models are already available here for chatglm3. https://huggingface.co/mradermacher/chatglm3-6b-GGUF/tree/main

We need a gguf model loader for chatglm3, so we can use it for Kolors. Thanks!

city96 commented 2 months ago

This repo currently only extends the unet and text encoder base nodes to support gguf models, so it would have to be natively supported in ComfyUI first for that to be possible.

To support a model like that we'd have to bundle llama.cpp in some form, not to mention the overhead of having to manage free vram between what would essentially two backends. With native support, we could just use the same logic as with T5 and create a mapping for the blocks to the ComfyUI format without much overhead.

As far as I can tell, this is the issue tracking it in ComfyUI: https://github.com/comfyanonymous/ComfyUI/issues/4113