search

getumbrel / llama-gpt

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
https://apps.umbrel.com/app/llama-gpt
MIT License
10.73k stars 696 forks source link

Is there a way to increase the token limit? #66

Closed timschwartz closed 1 year ago

timschwartz commented 1 year ago

I asked llama-gpt what the maximum token limit was and it said 1000, is that right?

mayankchhabra commented 1 year ago

Right now its 4096 tokens. Llama 2 models support max 4096 context length, and Code Llama models support 100K context length.