Access kcpp api for llava from comfyui somehow?

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

https://github.com/lostruins/koboldcpp

GNU Affero General Public License v3.0

4.99k stars 349 forks source link

Open merc4derp opened 6 months ago

merc4derp commented 6 months ago

We got nodes that support ollama endpoints for this task in comfyui. But ollama doesn't do clblast for older amd gpus.

Can we have an ollama compatible endpoint in kcpp or someone make a comfy node that works with the existing kcpp a1111-compatible endpoint maybe?

henk717 commented 6 months ago

If they support GPT4Vision give that a try, we emulate that one.