-
Hello, dear developer mkiol. Are you planning to add OpenAI Compatible API to your awesome application in the near future? If not, then I ask you to do so. In my opinion this would be incredibly u…
-
I tried Qwen2-72B-Instruct with both this quantization: https://huggingface.co/bartowski/Qwen2-72B-Instruct-GGUF/blob/main/Qwen2-72B-Instruct-Q4_K_M.gguf
And this one: https://huggingface.co/mraderma…
-
So following the recommended install method of cloning the client repository and running ./play-rocm.sh, the install goes smoothly then as soon as it starts to load kobold itself it immediately crashe…
-
Hello, can you add optional dynamic memory?
Record the input and output in a json file, and use the RAG method to store the memory in the koboldcpp backend.
-
Can someone please tell me, why the size of context in VRAM grows so much with layers? For example, if I have a model in GGUF with exactly 65 layers, then
```
.\koboldcpp.exe --model .\MLewd-ReMM-…
-
Hi , I'm using koboldcpp on Android through termux . I built it with openblas and clblast with make llama_openlas llama_clblast . Whenever I use --useclblast I get error low vram , I've experimented w…
-
Please make it so you can select what GPUs you want to use. Also I have Nvidia P40 and K80 I would like to be able to run them both. It defaults to my P40 and won't use my K80, if I disable my P40 it …
-
Очень неплохо. В принципе это то, что я хотел. Введённый текст на русском виден в окне диалога, а в промпт уходит английский перевод. Ответы ИИ на английском можно переводить даже динамически - плагин…
-
There are no 'install' instructions in cmake files.
It has only built 3 static libs: libcommon2.a libgpttype_adapter.a.
What can users do with these static libs?
Shouldn't there be any execut…
-
llama.cpp has an override-kv option that can be used to override, well, model kv values. This can be useful with the myriad of existing ggufs that don't have a pretokenizer specified. It would be nice…