git clone https://github.com/continuedev/ggml-server-example
cd ggml-server-example
python3 -m venv env
source env/bin/activate
on Mac, env\Scripts\activate.bat
on Windows, source env/bin/activate.fish
if using fish terminalpip install -r requirements.txt
models/
folder
python3 -m llama_cpp.server --model models/wizardLM-7B.ggmlv3.q4_0.bin
~/.continue/config.json
either manually or using the /config
slash command in Continue. Then, import the GGML
class (from continuedev.src.continuedev.libs.llm.ggml import GGML
), set "default_model": "default=GGML(max_context_length=2048)"
, reload your VS Code window, and you're good to go!Happy to help. Email use at hi@continue.dev.