gkamradt / langchain-tutorials

Overview and tutorial of the LangChain Library
6.65k stars 1.92k forks source link

Question on Pinecone index in 'Ask A Book Questions.ipynb' #4

Closed shaun0024 closed 1 year ago

shaun0024 commented 1 year ago

Hey Gregory,

Thank you for the great series on YouTube. I have a question regarding the notebook 'Ask A Book Questions.ipynb' that you used to demonstrate querying some custom knowledge from PDF files.

In the 11th cell, you used a code to load the vectors into Pinecone:

docsearch = Pinecone.from_texts([t.page_content for t in texts], embeddings, index_name=index_name)

Subsequently, you used docsearch again in your query:

query = "What are examples of good data science teams?"
docs = docsearch.similarity_search(query, include_metadata=True)

My question is would this be using the index from Pinecone? In your example here, you've loaded the vector into Pinecone earlier so the data is already in docsearch but for a use case where you would want to read the index directly without loading any documents from Pinecone, would you use from_existing_index instead? E.g.:

docsearch = Pinecone.from_existing_index(pinecone_index_name, embeddings)
gkamradt commented 1 year ago

Hey Shaun - thanks for the question!

Yes, this is using the index from Pinecone. When you call docsearch.similarity_search you are

  1. Sending your query to get a embedding from it
  2. Using that vector and running a similarity search against it with your Pinecone embeddings
  3. Getting the docs back and ready to use with LangChain/OpenAI for questioning
shaun0024 commented 1 year ago

Thanks for your reply. I think I need to reframe my question.

Based on this notebook, it is used to analyse documents and upload it as a vector into the Pinecone index.

So in the future, let's say I have a chat application that I want to build, I would just need to load the index from Pinecone:

docsearch = Pinecone.from_existing_index(pinecone_index_name, embeddings)

I've tried this and it appears to be working.

vbalance commented 1 year ago

Hey! Thank you for your job! It looks amazing! Would you mind helping me with the dependencies of the Ask A Book Questions.ipynb ?

I mean "!pip install [modules]" Thank you a lot