-
Logs as under
```
Welcome to KoboldCpp - Version 1.63
For command line arguments, please refer to --help
***
Attempting to use OpenBLAS library for faster prompt ingestion. A compatible libopen…
-
First, I'd like to say that the runpod image you created for koboldcpp is amazing, I'm currently using it. But for larger models, container pods are just too expensive for me, so I am experimenting wi…
-
Documentation seems to conflict over whether Rep Range being set to 0 or -1 disables range being used, testing with lite.koboldai UI, -1 automatically corrects to 0, and looking at code, 0 completely …
-
Hello,
Ollama does not support Windows per their website. Please add additional Inference we can use, if possible Koboldcpp (https://github.com/LostRuins/koboldcpp/)
-
***
Welcome to KoboldCpp - Version 1.68
For command line arguments, please refer to --help
***
Exception in thread Thread-7:
Traceback (most recent call last):
File "threading.py", line 932, i…
-
There may be a bug in the koboldcpp colab. I tried using a vision model like Llava 7B, but when I load one image, the processing is really slow. I noticed that the clip is being loaded on the CPU inst…
-
### 🚀 The feature, motivation and pitch
There are more guides showing up for ROCM on Windows such as this cuda program which needed cublas dependencies compiled with AMDs equivalent HIPblas:
https:/…
-
3xTesla P40, Llama-70B-q6, koboldcpp benchmark:
1.66.1 - prompt processing 8k = 82.44 sec, generation speed = 6.85 t/s
1.67 - prompt processing 8k = 81.60 sec, generation speed = 6.28 t/s
Prompt …
-
```
Processing Prompt (1 / 1 tokens)
Generating (1 / 242 tokens)
(EOS token triggered! ID:2)
CtxLimit: 3768/4096, Process:0.44s (442.0ms/T = 2.26T/s), Generate:0.00s (3.0ms/T = 333.33T/s), Total:0…
-
```
===> Building for koboldcpp-1.57.1
[ 1% 4/64] cd /usr/ports/misc/koboldcpp/work/koboldcpp-1.57.1 && /usr/local/bin/cmake -DMSVC= -DCMAKE_C_COMPILER_VERSION=16.0.6 -DCMAKE_C_COMPILER_ID=Clang -…