search

ian-kent / gptchat

A GPT-4 client which gives your favourite AI a memory and tools for self-improvement
MIT License
351 stars 51 forks source link

long-term memory is expensive. #11

Open punksta opened 1 year ago

punksta commented 1 year ago

Hi, amazing work!

As I can tell, long term memory requires sending all memories to gpt-4 to fetch the more relevant memories, for each input.

If it stoes many memories, the cost of this feature will be very high.

Is there a way of optimizing this? I am thinking about using a cheaper model.

ian-kent commented 1 year ago

Hey 👋 thanks!

I'm looking at options for improving this but I haven't had a lot of time to work on it. There's definitely optimisations that can be made, but any pre-filtering done on the data (which ultimately is the only solution) means letting something that isn't GPT determine how relevant a memory is.

It's worth trying with GPT-3.5 for memory recall though (you can replace openai.GPT4 in the memory module code), other parts of GPTChat (commands, plugins) don't work particularly well but memory recall might be ok.

Suppenterrine commented 1 year ago

Hello there! I saw your reddit post the other day where you mentioned this project. In the comments below I found the hint to use Vector Databases for Memory Storing. I'm coding a kind of similiar project like you, but in Javascript. I just wanted to share this video to you, because it helped me to integrate a vector db (I never worked with on before). Maybe it'll be useful to you: https://www.youtube.com/watch?v=tp0bQNDtLPc

And don't quit your project, it's a perfect way of learning because it keeps you motivated.