gustavz / DataChad

Ask questions about any data source by leveraging langchains
https://datachad.streamlit.app/
Apache License 2.0
314 stars 80 forks source link
activeloop chatbot chatgpt chatwithanything chatwithpdf embeddings knowledge-base langchain openai python streamlit

DataChad V3🤖

This is an app that let's you ask questions about any data source by leveraging embeddings, vector databases, large language models and last but not least langchains

How does it work?

  1. Upload any file(s) or enter any path or url to create Knowledge Bases which can contain multiple files of any type, format and content and create Smart FAQs which are lists of curated numbered Q&As.
  2. The data source or files are loaded and splitted into text document chunks
  3. The text document chunks are embedded using openai or huggingface embeddings
  4. The embeddings are stored as a vector dataset to activeloop's database hub
  5. A langchain is created consisting of a custom selection of an LLM model (gpt-3.5-turbo by default), multiple vector store as knowledge bases and a single special smart FAQ vector store
  6. When asking questions to the app, the chain embeds the input prompt and does a similarity search in in the provided vector stores and uses the best results as context for the LLM to generate an appropriate response
  7. Finally the chat history is cached locally to enable a ChatGPT like Q&A conversation

Good to know

How does it look like?

TODO LIST

If you like to contribute, feel free to grab any task