search

getumbrel / llama-gpt

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
https://apps.umbrel.com/app/llama-gpt
MIT License
10.73k stars 696 forks source link

Llama GPT extremely slow on Umbrel Home #149

Open anthony-robin opened 7 months ago

anthony-robin commented 7 months ago

I just set up my Umbrel Home and wanted to test self hosting Llama GPT on it. I was very astonished to see how slow it was to reply, even with a simple question (sometimes it never reply at all).

I thought Umbrel Home had an optimized hardware to make it runs smoothly ? Looking at the memory, the app only use 115,5MB RAM whereas it should use 5GB RAM. How can it be so low ?

I found this community post that reference a similar issue but still unresolved: https://community.umbrel.com/t/llamagpt-slow-on-home/13373

Thank you for helping me understand what is going one with a fresh new Umbrel home installation :)

shubhankartrivedi commented 6 months ago

issue is still there