Closed LostRuins closed 1 year ago
converting to draft PR pending further review.
Added a basic linux binary, which should be automatically selected on linux based OS.
For legal reasons and self compilability the PR should bundle koboldcpp's source code.
agreed, let me rework this such that it can function as a submodule pulled from the koboldcpp repo
Please test and review before merging. This is very much a proof-of-concept.
This PR adds KoboldCpp as a modular GGML backend to KoboldAI. It does not use the API, instead interfaces with KoboldCpp.py's python bindings to directly invoke cpp functions. Bundles KoboldCpp as prebuilt windows dlls. This is the first MVP, without additional customizability or accelerators included.
To use, this is an example launch command:
The model name expects the text
ggml
inside the filename.