search

getumbrel / llama-gpt

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
https://apps.umbrel.com/app/llama-gpt
MIT License
10.53k stars 666 forks source link

Too heavy for pi 4B #123

Closed jli113 closed 7 months ago

jli113 commented 8 months ago

The whole system stopped responding, llama-gpt with 7B is too much for 8G RAM Pi 4B.

ben-barnard commented 7 months ago

Works for me exactly at the speed listed in the documentation with 10 other applications installed. I would check your sdcard and SSD.

jli113 commented 7 months ago

Maybe using Zephyr or Mistral in the future?