-
**Describe the bug**
When trying to connect the program to koboldcpp's API (localhost:5001/v1) it throws an error.
**To Reproduce**
Install infinite-sides as normal, then launch a .gguf model wit…
-
### Feature Area
AI Services
### Painpoint
Unable to use KoboldCpp's URL for local LLM.
### Describe your idea
I'd like to recommend adding KoboldCpp to the "Run on Your Machine" section. This wo…
-
**_IMPORTANT NOTE: It's not Cuda12 at all, it's something wonky with my Kobold settings, they don't save!_**
**update:**
Hey, I found something super strange. I don't think it's Cuda at all but…
-
hello there, here is output when installing
```g++ -I. -I./common -I./include -I./include/CL -I./otherarch -I./otherarch/tools -I./otherarch/sdcpp -I./otherarch/sdcpp/thirdparty -I./include/vulkan …
-
I modified the port in "ReaperEngine.py" to be able to work with koboldcpp, however nothing happens after the text generation has finished.
Video is here:
https://github.com/Sebby37/Dead-Internet/as…
-
### Have you searched for similar requests?
Yes
### Is your feature request related to a problem? If so, please describe.
There's a ton of samplers that are implemented for the horde API but aren't…
-
When I put command ~/koboldcpp $ python koboldcpp.py --model KobbleTiny-Q4_K.gguf
I get an error *** Welcome to KoboldCpp - Version 1.66.1 Warning: OpenBLAS library file not found. Non-BLAS library…
-
### What happened?
Qwen2-72B-Instruct Q4_K_M generates output with random tokens (numbers, special symbols, random chunks of words from different languages, etc).
Has been tested on:
1) Tesla P…
-
### Your current environment
I can not find any doc about this feature.
### How would you like to use Aphrodite?
_No response_
-
Being able to see probability on the kobold API backend would be a nice addition