Lack of logging in the langchain_aws package

The langchain_aws package, in particular the ChatBedrockConverse class, has no standard python logging in it. This means visibility into issues is pretty dramatically reduced, and one has to either log at the boto3 level or across into AWS cloud resources. It would be useful to add more common python logging to the various classes in the langchain_aws package.

As an example real use case, I want to see the body of any request and responses sent to boto so I can understand how the language model might be interpreting some of the data. I expected I would see them if I did something like:

logging.getLogger('langchain_aws').setLevel(logging.DEBUG)

However, that doesn't do anything because the module doesn't log, and the best I could do was:

logging.getLogger('boto3').setLevel(logging.DEBUG)

This shows me request headers (not body, which is a problem), and responses (including body, yay!)

@cab938 LangChain provides some built-in utilities to log application data, that doesn't require adding logging statements directly to the integration classes.

LangSmith is the best tool for reviewing/monitor your langchain application. Here is the setup guide.

Use the set_debug flag. Here is an example:

from langchain_core.globals import set_debug
set_debug(True)

llm = ChatBedrockConverse(model="anthropic.claude-3-haiku-20240307-v1:0")
llm.invoke("What is the capital of France?")

# Output
[llm/start] [llm:ChatBedrockConverse] Entering LLM run with input:
{
"prompts": [
  "Human: What is the capital of France?"
]
}
[llm/end] [llm:ChatBedrockConverse] [261ms] Exiting LLM run with output:
{
"generations": [
  [
    {
      "text": "The capital of France is Paris.",
      "generation_info": null,
      "type": "ChatGeneration",
      "message": {
        "lc": 1,
        "type": "constructor",
        "id": [
          "langchain",
          "schema",
          "messages",
          "AIMessage"
        ],
        "kwargs": {
          "content": "The capital of France is Paris.",
          "response_metadata": {
            "ResponseMetadata": {
              "RequestId": "e411108a-0918-4d28-8030-a9b16f7c8ea1",
              "HTTPStatusCode": 200,
              "HTTPHeaders": {
                "date": "Mon, 21 Oct 2024 19:00:52 GMT",
                "content-type": "application/json",
                "content-length": "212",
                "connection": "keep-alive",
                "x-amzn-requestid": "e411108a-0918-4d28-8030-a9b16f7c8ea1"
              },
              "RetryAttempts": 0
            },
            "stopReason": "end_turn",
            "metrics": {
              "latencyMs": 220
            }
          },
          "type": "ai",
          "id": "run-15c8eb2f-7b15-44cd-8544-809ca1692036-0",
          "usage_metadata": {
            "input_tokens": 14,
            "output_tokens": 10,
            "total_tokens": 24
          },
          "tool_calls": [],
          "invalid_tool_calls": []
        }
      }
    }
  ]
],
"llm_output": null,
"run": null,
"type": "LLMResult"
}

Use callbacks. https://python.langchain.com/docs/how_to/callbacks_runtime/

langchain-ai / langchain-aws

Lack of logging in the langchain_aws package #245