deepset-ai / haystack

:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

https://haystack.deepset.ai

Apache License 2.0

14.49k stars 1.7k forks source link

Add the memory feature to the library to enable chat + RAG #7626

Open mrm1001 opened 2 months ago

mrm1001 commented 2 months ago

Add a memory abstraction to keep track of messages when running a chat pipeline that interacts with a retriever.

User story

Users should be able to create a pipeline that can answer user queries by using previous queries, responses (raw messages), as well as documents from a document store.

It needs to support the following conversation patterns:

App has access to previous content to understand context.

User: what is the tallest building in the US?
App: The North Tower.
User: How tall is it?
App: It is 417m long.

Outcome

Colab showcasing Chat + RAG.

### Tasks
- [ ] https://github.com/deepset-ai/haystack/issues/7868
- [ ] https://github.com/deepset-ai/haystack/issues/7917
- [ ] https://github.com/deepset-ai/haystack/issues/7918
- [ ] https://github.com/deepset-ai/haystack/issues/7919
- [ ] https://github.com/deepset-ai/haystack-cookbook/issues/97

mrm1001 commented 2 weeks ago

There were some issues that were identified with the current proposed approach: https://github.com/deepset-ai/haystack-cookbook/pull/90#issuecomment-2161170220.

The two key issues today are:

user messages are not tracked in memory
the use of the outputAdapter component is bad for UX

We will gather requirements about how we want to be able to use the memory, and add them to this issue.

mrm1001 commented 1 week ago

As a dev, I would like to:

add chat messages to the memory from different parts of my graph (e.g. query input and from output of LLM) while keeping the additional number of components needed low
have flexibility to store chat messages without the need for any conversion to documents. Ideally no conversion at all so that the memory can save chat messages directly.

julian-risch commented 1 week ago

@vblagoje Let's define what needs to be done in this issue together today. I have in mind breaking it up into four smaller issues and we can split the work.

Add InMemoryChatMessageStore in haystack-experimental
Add ChatMessageRetriever in haystack-experimental
Add ChatMessageWriter in haystack-experimental
Add user messages to memory in colab example notebook

CarlosFerLo commented 2 days ago

Heyy, have a look at this issue #7830 where an abstraction for indexable objects is proposed. We could make ChatMessage inherit from Indexable and just by writing once the code for ObjectWriter and for the different ObjectRetriver components we can apply them to this use case.