search

keldenl / gpt-llama.cpp

A llama.cpp drop-in replacement for OpenAI's GPT endpoints, allowing GPT-powered apps to run off local llama.cpp models instead of OpenAI.
MIT License
594 stars 67 forks source link

"Add GPU layer offload option. defaults.js" #47

Closed jnchman closed 1 year ago

jnchman commented 1 year ago

Adds new GPU layer offload argument available in the latest version of llama.cpp

keldenl commented 1 year ago

just did this! and added support for shortform flag too (ngl). thanks! (https://github.com/keldenl/gpt-llama.cpp/commit/942ed88773a9eb7cc8543d6d85a69695db893fc1)