Bump llama-cpp-python to support GGUF models and add very basic "unknown" model support

edgar971 / open-chat

A self-hosted, offline, ChatGPT-like chatbot with different LLM support. 100% private, with no data leaving your device.

MIT License

64 stars 8 forks source link

Bump llama-cpp-python to support GGUF models and add very basic "unknown" model support #11

Open raiju opened 9 months ago

raiju commented 9 months ago

I don't have the bandwidth/time to make this a full PR (particularly the unknown model support implies better config being required), but putting this up in case it's helpful (particularly the breaking llama.cpp upgrade).

You can also test this on unraid directly by replacing the repository of an existing install in unraid with ghcr.io/raiju/open-chat-cuda:v2.0.0-alpha.6.