search

getumbrel / llama-gpt

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
https://apps.umbrel.com/app/llama-gpt
MIT License
10.73k stars 696 forks source link

Not enough space #146

Open gruzilla opened 7 months ago

gruzilla commented 7 months ago

When sending large prompts i get the following error:

ggml_allocr_alloc: not enough space in the buffer (needed 561259008, largest block available 102924288)
GGML_ASSERT: /private/var/folders/qj/w85147014pq9yp4k5j6g66tw0000gn/T/pip-install-5wl_01v9/llama-cpp-python_451859871b5d42ada35cd84d4620a385/vendor/llama.cpp/ggml-alloc.c:139: !"not enough space in the buffer"

Does this mean, my machine does not have enough RAM (can't veryify, there seems to be RAM left), or is there an option i can give this process more memory (can't find an option)?

thank you for your answer!

Haui1112 commented 6 months ago

Same issue here with the llama-2-7b-chat model.