search

getumbrel / llama-gpt

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
https://apps.umbrel.com/app/llama-gpt
MIT License
10.82k stars 699 forks source link

Add Code Llama support #71

Closed mayankchhabra closed 1 year ago

mayankchhabra commented 1 year ago

This pull request introduces support for Code Llama models with GGUF format. High level changes:

  1. New docker-compose-gguf.yml file: A separate configuration file has been created to handle the specific requirements of GGUF models.

  2. Updates to docker-compose.yml and docker-compose-mac.yml files: The existing configuration files have been modified to include configurations for the new models and existing GGML models.

  3. Modifications to run.sh and run-mac.sh files: The startup scripts have been updated to include support for the new models, and model-specific configurations, and improved model handling.