search

zmedelis / bosquet

Tooling to build LLM applications: prompt templating and composition, agents, LLM memory, and other instruments for builders of AI applications.
https://zmedelis.github.io/bosquet/
Eclipse Public License 1.0
280 stars 19 forks source link

brainstorm memory #23

Closed behrica closed 1 year ago

behrica commented 1 year ago

to continue from #22 regarding memory:

After I have read here: https://archive.pinecone.io/learn/langchain-conversational-memory/

I see know that there are conceptually two very different types of memory.

  1. The article above talks about "conversational memory", so a memory which lives only during a longer lasting conversation between a person and a LLM.

  2. What I have tried is more about giving an LLM access to a "long term persistent memory"

    • a search for semantic similar text snippets to a question and then call gen/complete-template ones for each snippet found
    • a extra gen/complete-template which takes in results of above and and other instructions to summarize this (indeed, paying attention of not passing token limit)

Both might / might not rely on vector databases.

Both have in common that gen/complete-template is called several times:

  1. with user interaction between the calls
  2. without user interaction between the calls
behrica commented 1 year ago

I had an idea on the conversational memory in functional terms, which is maybe worth to discuss. A "conversation" could be seen as various calls to gen/complete-template where in every call some from of user supplied call back fn can be triggered. This callback-fn could serve 2 purposes:

  1. it gives somehow access to all slots and model-opts of the last calls (the inputs to and outputs from teh model / the rendered template)
  2. it allows to modify the next call based on this history

I am not sure yet, how to express this with means of Clojure

behrica commented 1 year ago

The "example of this" is a user supplied call back fn, which accumulates previous answers in the context up to a certain length (the max allowed tokens), from which onwards it would need to "summarize" via an extra call to a LLM

zmedelis commented 1 year ago

I am on vacation. Back next week and happy to discuss.

zmedelis commented 1 year ago

I have moved out conversation mode part to this issue #25

zmedelis commented 1 year ago

Closing this with the latest release supporting memory handling. See https://zmedelis.github.io/bosquet/notebook/examples/short_memory_prosocial_dialog/index.html will be adding long term memory documentation soon. This is a complex topic and memory handling will evolve