search

gokayfem / ComfyUI_VLM_nodes

Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation
Apache License 2.0
414 stars 36 forks source link

[Feature Request] Support for GPTQ Model #8

Closed nomadoor closed 9 months ago

nomadoor commented 9 months ago

The 4-bit quantized version of InternLM-XComposer2-VL, known as InternLM-XComposer2-VL-7b-4bit, which has demonstrated excellent performance, has been released.

I would be very pleased if this could be supported. However, it might be better not to cram too much into a single repository.

gokayfem commented 9 months ago

I'm planning to add it, currently working on outputs of the VLM and other nodes, it can take some time

nomadoor commented 9 months ago

Thank you for the wonderful job! I’m looking forward to it.

gokayfem commented 9 months ago

i added this model to repo 👍

nomadoor commented 9 months ago

It’s working perfectly. Thank you so much for the wonderful job😍 However, there is one thing that concerns me.

Some custom nodes like comfyui_controlnet_aux have started to fail on import. It shows ImportError: Matplotlib requires dateutil>=2.7; you have 2.2, but python-dateutil-2.8.2 is already installed. I tried pip install python-dateutil==2.7 and it started working normally, but is this related to VLM_nodes?

gokayfem commented 9 months ago

yes its related to that i fixed it as you suggest, thank you 👍

nomadoor commented 9 months ago

Thank you!!

gokayfem commented 9 months ago

i also added kosmos-2 and UForm-Gen2Qwen(This model is blazing fast like joytag but extremly good almost llava 1.5 level)

nomadoor commented 8 months ago

That’s great news, I tried it out right away! UForm-Gen2Qwen isn’t bad, but considering the VRAM usage, I prefer llava-v1.6-mistral-7b which uses about the same amount. Also, joytag, which hardly monopolizes VRAM, is still tier 1 even now.

gokayfem commented 8 months ago

definitely llava 1.6 mistral number one choice by far