Chroma search with vector and search with text get different result using the same embedding function

alphrc commented 3 weeks ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[x] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

embedding_function = OpenAIEmbeddings(model="text-embedding-3-large", api_key=os.getenv("OPENAI_API_KEY"))
db = Chroma(persist_directory=directory, embedding_function=embedding_function, collection_name='default')

query = "Some text for query"

# Search with text
results_text = db.similarity_search(query)

# Search with vector
vector = embedding_function.embed_documents([query])[0]
results_vector = db.similarity_search_with_vector(vector)

Error Message and Stack Trace (if applicable)

No response

Description

Using the same embedding function, searching with text and searching with vector would get different results.

System Info

langchain==0.2.11 langchain-chroma==0.1.2 langchain-community==0.2.10 langchain-core==0.2.23 langchain-openai==0.1.17 langchain-text-splitters==0.2.2

max

python 3.9.6

alphrc commented 3 weeks ago

This issue remains even if you use embed_query instead of embed_documents

jhaayush2004 commented 3 weeks ago

It's not the bug in Langchain for sure as I explored Similarity_search as well as similarity_search_by_vector methods using HuggingFaceEmbeddings in Langchain framework itself and got the exactly same results from both.

db = Chroma.from_documents(texts,huggingface_embeddings)
query = "what happened to harry's parents ?"
results_text = db.similarity_search(query)

vector = huggingface_embeddings.embed_documents([query])[0]
results_vector = db.similarity_search_by_vector(vector)

if (results_text == results_vector):
  print("True")
else:
  print("False")

The output which came was "True" .

Issue which might be causing problem:

There is no method named as similarity_search_with_vector in langchain_chroma integrational framework , so replace it with similarity_search_by_vector .
There might be some issue in langchain_openai integration but chances of that are very less . I can't test it because I don't have access to OpenAI paid version.

langchain-ai / langchain