search

getumbrel / llama-gpt

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
https://apps.umbrel.com/app/llama-gpt
MIT License
10.53k stars 666 forks source link

Added Llama 3 8B support #156

Open Ualas opened 2 months ago

Ualas commented 2 months ago

This pull request introduces support for Llama 3 8B model with GGUF format. The high-level changes are:

  1. Updated llama-cpp-python to v0.2.63. The previous version was causing the API to crash.
  2. Changed DEFAULT_MODEL to Llama 3 8B.
  3. Added API and UI compatibility to the Llama 3 8B model.
gyzerok commented 2 months ago

@Ualas would it make sense to add 70b as well?