ConversationalRetrievalChain having trouble with vector store retriever 'as_retriever()'

[kanik9]

System Info

Code snapshot:

Data in Azure Cognitive Search Vector Store: data ={ "id": " ", "content": "", "content_vector": [], "metadata": "{}", "a": "", "b": "", "c": 20.4, "d": "" }

Issue: with langchain==0.0.261, but it is working fine with langchain==0.0.242 Issue Description : SerializationError: (', DeserializationError: (", AttributeError: \'float\' object has no attribute \'lower\'", \'Unable to deserialize to object: type\', AttributeError("\'float\' object has no attribute \'lower\'"))', 'Unable to build a model: (", AttributeError: \'float\' object has no attribute \'lower\'", \'Unable to deserialize to object: type\', AttributeError("\'float\' object has no attribute \'lower\'"))', DeserializationError(", AttributeError: 'float' object has no attribute 'lower'", 'Unable to deserialize to object: type', AttributeError("'float' object has no attribute 'lower'")))

Note: In the upgraded version the response time also increases.

Who can help?

No response

Information

• [ ] The official example notebooks/scripts
• [X] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [X] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [X] Vector Stores / Retrievers
• [X] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [X] Chains
• [X] Callbacks/Tracing
• [ ] Async

Reproduction

def __llm_client(self, streaming: bool =False):

LLM connection

    if streaming is False:
        return AzureOpenAI(
            deployment_name=AZURE_OPENAI_LLM_MODEL_DEPLOYMENT_NAME,
            model=AZURE_OPENAI_LLM_MODEL_NAME,
            openai_api_type=self.__azure_openAI_type,
            openai_api_version=self.__azure_openAI_version,
            openai_api_base=self.__azure_openAI_base,
            openai_api_key=self.__azure_openAI_key,
            temperature=0.0,
            max_tokens=2000
        )
    return AzureOpenAI(
            deployment_name=AZURE_OPENAI_LLM_MODEL_DEPLOYMENT_NAME,
            model=AZURE_OPENAI_LLM_MODEL_NAME,
            openai_api_type=self.__azure_openAI_type,
            openai_api_version=self.__azure_openAI_version,
            openai_api_base=self.__azure_openAI_base,
            openai_api_key=self.__azure_openAI_key,
            temperature=0.0,
            max_tokens=2000,
            streaming=True, 
            callbacks=[StreamingStdOutCallbackHandler()]
        )

# LLM Embeddings Client
def __embeddings_client(self):
    return OpenAIEmbeddings(
        model=AZURE_OPENAI_LLM_EMBEDDING_NAME,
        deployment=AZURE_OPENAI_LLM_EMBEDDING_DEPLOYMENT_NAME,
        openai_api_type=self.__azure_openAI_type,
        openai_api_version=self.__azure_openAI_version,
        openai_api_base=self.__azure_openAI_base,
        openai_api_key=self.__azure_openAI_key,
        chunk_size=1536
    )

# Embedding vector store client
def __vector_store_client(self):
    acs_vector_store: AzureSearch = AzureSearch(
        azure_search_endpoint=self.__acs_endpoint,
        azure_search_key=self.__acs_key,
        index_name=self.__acs_index_name,
        embedding_function=self.__embeddings_client().embed_query,
    )
    return acs_vector_store

# Langchain Chain Client
def __chain_client(self):
    chain_type = "stuff"
    return load_qa_chain(
        llm=self.__llm_client(streaming=True), 
        chain_type=chain_type,
        document_variable_name='context',
        prompt=QA_PROMPT
        # verbose=True
        )

def __conversational_retrieval_chain_client(self):
    # print(self.__vector_store_client().as_retriever())
    return ConversationalRetrievalChain(
        retriever=self.__vector_store_client().as_retriever(),
        question_generator=self.__question_generator(),
        combine_docs_chain=self.__chain_client(),
        memory=ConversationBufferMemory(
            memory_key="chat_history",
            return_messages=False
        )
    )

def __question_generator(self):
    return LLMChain(
        llm=self.__llm_client(),
        prompt=CONDENSE_QUESTION_PROMPT
    )

# Main function
def smart_chat_bot(self, query: str="*", conversation: list=[]):
    self.user_input = query
    print(f"Human Input: {self.user_input}", end="\n")
    result = self.__conversational_retrieval_chain_client()({"question": self.user_input, "chat_history": conversation})
    return result

Expected behavior

It should return the response on the bases of user input and the documents that came from the vector store. And the latency will be lesser.

[dosubot[bot]]

Answer generated by a 🤖

Answer

Hello,

Thank you for providing detailed information about the issue you're facing. The error message you're seeing, AttributeError: 'float' object has no attribute 'lower', suggests that there's a part of the code that's trying to call the lower() method on a float object. This is likely causing the SerializationError you're encountering.

From the code you've shared, it's not immediately clear where this might be happening. However, given that the error occurs when you're using the as_retriever() method, it's possible that the issue lies in how the data is being processed or serialized within this method or the objects it interacts with.

As for the increased response time in the newer version, it could be due to additional processes or checks introduced in the newer version. Without specific details on the changes made between versions 0.0.242 and 0.0.261, it's hard to pinpoint the exact cause.

To help resolve this issue, could you please provide more information on the following:

1. The structure of the data you're passing to the as_retriever() method. Specifically, are there any float values that might be mistakenly treated as strings?
2. Any changes you've noticed in the as_retriever() method or related components between LangChain versions 0.0.242 and 0.0.261.

This additional information will help us understand the issue better and provide a more accurate solution.

Best, Dosu

Sources

• libs/langchain/tests/integration_tests/chains/test_retrieval_qa.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[ghostenginesio]

Finding the same issue. Vectorstore retriever is not sending the correct documents in many cases. Sometimes it does work. Why is this the case?

[dosubot[bot]]

Hi, @kanik9,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. The issue you raised pertains to a SerializationError encountered when using the ConversationalRetrievalChain with the vector store retriever in langchain library version 0.0.261, resulting in increased response time compared to version 0.0.242. I have provided a response suggesting that the error may be related to a lower() method being called on a float object and have requested more information about the data structure and changes between the versions to further investigate the issue. Additionally, user has reported experiencing a similar problem with the vector store retriever not sending the correct documents in some cases.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days.

Thank you!

[kanik9]

Hey, thanks for the suggestion, but I'm not currently working on this problem. When I start working on something related to the ConversationalRetrievalChain, I will test this and share the findings with you.