Title: Dynamic FAISS Integration for RAG in Prototyping Conversations
Description
Implement a lightweight Retrieval-Augmented Generation (RAG) solution using FAISS for small-scale prototyping scenarios. This feature will allow users to dynamically add local files (PDFs and text files) during an ongoing conversation, process embeddings and index them on-the-fly, and retrieve relevant document chunks to enrich dialog context.
The goal is to seamlessly integrate this functionality into the existing conversational flow, enabling rapid prototyping without requiring persistent storage or external services.
Acceptance Criteria
File Parsing
Extract text from user-provided PDFs and text files.
Split large documents into smaller, manageable chunks (e.g., 512 tokens).
Dynamic Indexing
Use FAISS to dynamically embed document chunks and add them to an in-memory index during the conversation.
Ensure embeddings are generated using a lightweight model like all-MiniLM-L6-v2.
Metadata Management
Maintain a dictionary to map FAISS indices to the corresponding text chunks for retrieval.
Query Handling
Implement a query system that retrieves the top-k most relevant document chunks from FAISS.
Concatenate retrieved chunks and feed them into the LLM for contextual generation.
Integration with Conversation
Enable users to add documents mid-conversation, with seamless processing and indexing.
Allow queries during the conversation to dynamically fetch and use the indexed content.
Tasks
File Parsing
[ ] Implement text extraction for PDFs using PyPDF2 or similar.
[ ] Handle text file loading and chunking into smaller segments.
FAISS Indexing
[ ] Set up an in-memory FAISS index (IndexFlatL2).
[ ] Embed document chunks using SentenceTransformer.
Metadata Tracking
[ ] Create and maintain a metadata dictionary linking FAISS indices to text chunks.
Query System
[ ] Implement querying to retrieve the top-k relevant chunks from FAISS.
[ ] Concatenate retrieved chunks into a context string for LLM integration.
Conversation Integration
[ ] Allow dynamic file uploads during a session, updating the FAISS index and metadata.
[ ] Extend query logic to incorporate RAG responses seamlessly into the ongoing dialog.
Notes
This feature is designed for prototyping and assumes a small dataset that fits in memory.
Persistent storage for FAISS and metadata is not required for this implementation.
Future enhancements (e.g., database-backed storage or cloud integration) can build upon this foundation.
This feature is crucial for enabling dynamic document handling and retrieval in prototyping conversations and will form the potential backbone for future RAG implementations.
rluijk
commented
3 hours ago
Title: Dynamic FAISS Integration for RAG in Prototyping Conversations
Description
Implement a lightweight Retrieval-Augmented Generation (RAG) solution using FAISS for small-scale prototyping scenarios. This feature will allow users to dynamically add local files (PDFs and text files) during an ongoing conversation, process embeddings and index them on-the-fly, and retrieve relevant document chunks to enrich dialog context.
The goal is to seamlessly integrate this functionality into the existing conversational flow, enabling rapid prototyping without requiring persistent storage or external services.
Acceptance Criteria
File Parsing
Dynamic Indexing
all-MiniLM-L6-v2.Metadata Management
Query Handling
Integration with Conversation
Tasks
File Parsing
PyPDF2or similar.FAISS Indexing
IndexFlatL2).SentenceTransformer.Metadata Tracking
Query System
Conversation Integration
Notes
References
Priority: High
This feature is crucial for enabling dynamic document handling and retrieval in prototyping conversations and will form the potential backbone for future RAG implementations.