langchain-ai / langchain

🦜🔗 Build context-aware reasoning applications
https://python.langchain.com
MIT License
94.2k stars 15.23k forks source link

AmazonKnowledgeBasesRetriever breaks application. When using KnowledgeBase directly with Boto3 no error. #16686

Closed KBB99 closed 9 months ago

KBB99 commented 9 months ago

Checked other resources

Example Code

from langchain_community.retrievers import AmazonKnowledgeBasesRetriever
retriever = AmazonKnowledgeBasesRetriever(
        knowledge_base_id="<knowledge_base_id>"
        retrieval_config={"vectorSearchConfiguration": {"numberOfResults": 3}}
    )
retriever_query = "TEST"
documents = retriever.get_relevant_documents(query=retriever_query)

Error Message and Stack Trace (if applicable)


documents = retriever.get_relevant_documents(query=retriever_query)
  File "/opt/python/langchain_core/retrievers.py", line 200, in get_relevant_documents
    callback_manager = CallbackManager.configure(
  File "/opt/python/langchain_core/callbacks/manager.py", line 1400, in configure
    return _configure(
  File "/opt/python/langchain_core/callbacks/manager.py", line 1947, in _configure
    logger.warning(
  File "/var/lang/lib/python3.12/logging/__init__.py", line 1551, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/var/lang/lib/python3.12/logging/__init__.py", line 1684, in _log
    self.handle(record)
  File "/var/lang/lib/python3.12/logging/__init__.py", line 1700, in handle
    self.callHandlers(record)
  File "/var/lang/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
    hdlr.handle(record)
  File "/var/lang/lib/python3.12/logging/__init__.py", line 1028, in handle
    self.emit(record)
  File "/var/lang/lib/python3.12/site-packages/awslambdaric/bootstrap.py", line 303, in emit
    msg = self.format(record)
  File "/var/lang/lib/python3.12/logging/__init__.py", line 999, in format
    return fmt.format(record)
  File "/var/lang/lib/python3.12/logging/__init__.py", line 703, in format
    record.message = record.getMessage()
  File "/var/lang/lib/python3.12/logging/__init__.py", line 392, in getMessage
    msg = msg % self.argsEND RequestId: c9f27447-0d68-4100-bacf-e2bde27a72ab
    ```

### Description

When I use knowledge bases directly with Boto3 client it works. This makes me suspect error is coming from or callbackManager, but I don't know why that would be used?

for result in results:
            documents.append(
                Document(
                    page_content=result["content"]["text"],
                    metadata={
                        "location": result["location"],
                        "score": result["score"] if "score" in result else 0,
                    },
                )
            )

### System Info

From Lambda Layer Python3.12 Runtime with ARM64 architecture. Langchain version 0.1.4. Boto3 version 1.34.29.
dosubot[bot] commented 9 months ago

🤖

Hey @KBB99! Great to see you back here in the LangChain repo. Looks like you've caught another quite interesting bug. Let's dive into this together, shall we?

Based on the information provided, it seems like the issue might be related to the CallbackManager in the AmazonKnowledgeBasesRetriever class. The CallbackManager is used to handle various events that occur during the execution of the language model. It's designed to manage and handle callbacks from LangChain.

However, without the complete code of AmazonKnowledgeBasesRetriever, it's hard to pinpoint the exact issue. The error stack trace you provided indicates that the issue might be related to logging, as the error seems to occur when the CallbackManager tries to log a warning message.

One possible reason could be that the CallbackManager is not properly configured, or there might be an issue with the logging configuration in your environment. It's also possible that there's a bug in the CallbackManager or AmazonKnowledgeBasesRetriever class that's causing this issue.

To further investigate this issue, could you please provide the complete code of the AmazonKnowledgeBasesRetriever class and the configuration of your CallbackManager? This would help in identifying the exact cause of the issue.

Also, could you please provide more details about the error? For example, what is the exact error message? Does the error occur every time you run the code, or only under certain conditions? Any additional information would be helpful.

As for your question about why the CallbackManager is used in the AmazonKnowledgeBasesRetriever, it's used to handle events specific to that component. For example, if the AmazonKnowledgeBasesRetriever generates an event when it starts retrieving data, the CallbackManager would call the appropriate event handler to handle this event.

I hope this helps! If you have any more questions, feel free to ask.

Best regards, Dosu

Sources


This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

KBB99 commented 9 months ago

@dosu-bot When I set LANGCHAIN_TRACING_V2 to false it works. So it must be an issue with the way responses are being processed by Langsmith tracing.

dosubot[bot] commented 9 months ago

🤖

Based on the information provided, it seems like the issue you're encountering is related to the CallbackManager in the LangChain codebase. The CallbackManager is responsible for handling various events that occur during the execution of the language models. It provides a way to define custom behavior when these events occur, such as when a language model starts running, generates a new token, ends running, or encounters an error.

However, the provided context does not include any direct interaction between the CallbackManager and the AmazonKnowledgeBasesRetriever, so it's not possible to provide a detailed explanation of their interaction.

The error message you provided suggests that the issue might be related to the logging system. The CallbackManager uses Python's built-in logging module to log warnings and errors. The error message indicates that an error occurred while formatting a log message. This could be due to a problem with the log message itself (for example, if it contains a format specifier like %s but the corresponding argument is not provided), or with the logging configuration.

You mentioned that setting LANGCHAIN_TRACING_V2 to false resolves the issue. This suggests that the problem might be related to the tracing functionality in LangChain. Tracing is a technique used to monitor the execution of an application, which can be useful for debugging and performance optimization. It's possible that there's a bug in the tracing code that's causing this issue.

Unfortunately, without more information, it's difficult to determine the exact cause of the problem. If you could provide more details about the error, such as the full error message and stack trace, it would be easier to diagnose the issue. Additionally, if you could share the code that you're using to initialize and configure the CallbackManager and the AmazonKnowledgeBasesRetriever, it might help to identify any potential issues with the way they're being used.

For more information about the CallbackManager and its methods, you can refer to the LangChain codebase.

Sources


This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

KBB99 commented 9 months ago

Error is a bit misleading. Turns out it was failing, because I was missing the LANGCHAIN_API_KEY in environment variables.