search

wawawario2 / long_term_memory

A gradio web UI for running Large Language Models like GPT-J 6B, OPT, GALACTICA, LLaMA, and Pygmalion.
GNU Affero General Public License v3.0
307 stars 15 forks source link

Interesting Idea to increase usable context. #10

Open USBhost opened 1 year ago

USBhost commented 1 year ago

So I have been thinking. So Google's Gchat has a feature that summarizes past conversations, Could we use this idea? Like if the bot explained or said something long could we feed the conversations back as a separate session to get the summary and replace some of our context with smaller versions for LTM or the current Prompt?

Even if our current models do not summarize well future ones will. And if our max context size enlarges this feature will effectively make it larger still.

wawawario2 commented 1 year ago

I believe that's something langchain supports. It's a very good approach not just for the reasons you mentioned (managing our context budget) but because it'll also limit filler text that enters the system.