search

KoboldAI / KoboldAI-Client

https://koboldai.com
GNU Affero General Public License v3.0
3.45k stars 743 forks source link

Feature Request: Support Ollama API #444

Closed BrainSlugs83 closed 1 month ago

BrainSlugs83 commented 1 month ago

Ollama supports serving super fast quantized gguf versions of models, and if we're already invested in that eco system, and have hundreds of GB of models downloaded -- then having to redownload them is really annoying -- and there doesn't seem to be any support in Kobald for quantized models.

If you can add support for calling the ollama api -- where we just provide a hostname and a port, that would be amazing. Thank you.

henk717 commented 1 month ago

Won't be added, if you want to use a GGUF based solution you can use our https://koboldai.org/cpp which has full KoboldAI support.