Add context to chat bot

[mcnewcp]

Summary

Add a vectorized database for context to the chat bot. This can be done using a similarity search and appending context to the prompt, or by using a more direct QA retrieval chain from langchain.

Dataset

Some dataset idea resources:

• financial reddit data on huggingface
• and the repo used to create the dataset above, including rating, sorting, scoring, filtering
• reddit bulk downloader repo
• pushshift reddit archive. This one has been put on hold as of 03/23
• tutorial on huggingface datasets

Technique

Trial the following Retrieval Augmented Generation (RAG) techniques:

1. RetrievalQA (example 1, example 2, streamlit example) will execute one for one questions and answers on the embedded data source.
2. Add chat history - this builds on above by retaining history and allowing for a chat interaction (ConversationalRetrievalQA example, manual example).
3. Conversational Retrieval Agent (example 1, blog post) improves on ConversationalRetrievalQA by more efficiently using chat history and also choosing when to query the data source and when not to.
4. Retrieval with metadata. By including metadata in the index, the context can be improved by filtering on metadata prior to the semantic search. Many popular vector dbs support metadata filters, and LlamaIndex provides some tools for managing the metadata easily. The metadata filters can be applied manually, or generated by the LLM (Auto-Retrieval) or agent (OpenAI Agent + Query Engine).

[mcnewcp]

@907Resident FYI see above for a working summary of techniques to trial for question answering with context.

[907Resident]

Nice, I will try some of these ideas out locally. I should be able to get at least one looked at today

Thanks for sharing