feat: context sliding window

edgenai / edgen

⚡ Edgen: Local, private GenAI server alternative to OpenAI. No GPU required. Run AI models locally: LLMs (Llama2, Mistral, Mixtral...), Speech-to-text (whisper) and many others.

https://docs.edgen.co/

Apache License 2.0

323 stars 14 forks source link

feat: context sliding window #101

Open francis2tm opened 5 months ago

francis2tm commented 5 months ago

Right now, if a session reaches the max token context size, it seg faults. This cannot happen