Ollama supports serving super fast quantized gguf versions of models, and if we're already invested in that eco system, and have hundreds of GB of models downloaded -- then having to redownload them is really annoying -- and there doesn't seem to be any support in Kobald for quantized models.
If you can add support for calling the ollama api -- where we just provide a hostname and a port, that would be amazing. Thank you.
Ollama supports serving super fast quantized gguf versions of models, and if we're already invested in that eco system, and have hundreds of GB of models downloaded -- then having to redownload them is really annoying -- and there doesn't seem to be any support in Kobald for quantized models.
If you can add support for calling the ollama api -- where we just provide a hostname and a port, that would be amazing. Thank you.