What types of quants of LLMs does your node support?

StartHua / Comfyui_CXH_joy_caption

Recommended based on comfyui node pictures:Joy_caption + MiniCPMv2_6-prompt-generator + florence2

Apache License 2.0

444 stars 26 forks source link

Closed q5sys closed 2 months ago

q5sys commented 2 months ago

There are lots of quantization types, obviously bnb is supported, but what about other quant methods: gguf, gptq, awq, exl2, etc.

q5sys commented 2 months ago

nevermind I tested and it doesnt seem to support anything other than bnb.