search

redis-developer / ArXivChatGuru

Use ArXiv ChatGuru to talk to research papers. This app uses LangChain, OpenAI, Streamlit, and Redis as a vector database/semantic cache.
MIT License
520 stars 69 forks source link

Need explanation how redis search helps here #2

Closed FuncGuy closed 1 year ago

FuncGuy commented 1 year ago

@tylerhutcherson Can you please explain what is happening in brief? or any reference link for understanding in brief so far my understanding is at the app startup it will load csv into redis and when the user queries it will look into redis and construct some tokens[this part is not clear] and it fires request to openAI for the response. How redis search helps here

tylerhutcherson commented 1 year ago

Hi! If you checkout the project readme and the app UI when it loads, there's an architecture diagram and explanation.

Redis (via RediSearch) stores document embeddings. When you ask a question,

  1. we embed the question with OpenAI.
  2. we search redis for relevant document chunks (that help answer the question) using vector similarity search.
  3. we pipe the results OpenAI GPT to synthesize an actual answer.