search

LagPixelLOL / ChatGPTCLIBot

ChatGPT Bot in CLI with long term memory support using Embeddings.
MIT License
340 stars 38 forks source link

Cumulative Token Bloat? #14

Closed NeauraNightsong closed 1 year ago

NeauraNightsong commented 1 year ago

Hey so I've been playing around with this for about 30 minutes now and it seems I've already filled up its memory? I cannot send even short messages to it without it saying "Error when calling API: Max tokens exceeded in messages: 8207 >= 8191"

I don't know how it's supposed to store long term memories, if it can only hold a few before it's using up all the tokens on whatever's going on behind the scenes, it gets impossible to communicate with only a page or two of text worth of memory being stored in the save file thru interacting with it.

am I doing something wrong? Sorry for the questions if they're obvious or etc, just not understanding why I'm running into this problem when my prompts are very short now

image

LagPixelLOL commented 1 year ago

Your initial prompt is probably too long, the initial prompt is not the place to store the memory, please consider using document Q&A mode instead. Also you can't just feed entire pages of stuff into the prompt as this is not the correct way of doing it too. If you need to provide it large text files to reference, please use document Q&A mode. Also please read the wiki for more info as reading it is actually much easier and faster than asking in issues!

LagPixelLOL commented 1 year ago

Closed as no response.