ollama chat not realised

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

Apache License 2.0

741 stars 39 forks source link

ollama chat not realised #32

Open xldistance opened 3 months ago

xldistance commented 3 months ago

Error when using api chat with vscode's continue plugin

  File "E:\open-webui\backend\python311\Lib\site-packages\ktransformers\server\api\ollama\completions.py", line 96, in chat
    raise NotImplementedError
NotImplementedError

xldistance commented 3 months ago

When will ollama api chat and autocomplete be supported?

UnicornChan commented 3 months ago

In the past few days, we have been focusing on enhancing support for native Windows, multiple GPUs......, the next version we will support long context and kv cache store. So, in the following version, we will support the Ollama api chat.