langchain-ai / langchain

🦜🔗 Build context-aware reasoning applications
https://python.langchain.com
MIT License
88.33k stars 13.86k forks source link

get_bedrock_anthropic_callback does not return token values correctly #21732

Open cis-hotta opened 1 month ago

cis-hotta commented 1 month ago

Checked other resources

Example Code

from langchain_aws import ChatBedrock
from langchain_community.callbacks.manager import get_bedrock_anthropic_callback

llm = ChatBedrock(model_id="anthropic.claude-3-haiku-20240307-v1:0")
# llm = ChatBedrock(model_id="anthropic.claude-v2")

with get_bedrock_anthropic_callback() as cb:
    result = llm.invoke("Tell me a joke")
    result2 = llm.invoke("Tell me a joke")
    print(cb)

Error Message and Stack Trace (if applicable)

This is not an error, but rather the actual behavior. As shown on the referenced page -> Tracking token usage.

Tokens Used: 0
    Prompt Tokens: 0
    Completion Tokens: 0
Successful Requests: 2
Total Cost (USD): $0.0

Description

Description: I encountered an issue with the get_bedrock_anthropic_callback function in Langchain. According to the documentation, the function should provide token usage details, but it returns all token values as 0.

Steps to Reproduce:

Use get_bedrock_anthropic_callback as described in the documentation. Observe that token values are returned as 0. Expected Behavior: The function should return the correct token usage values.

Actual Behavior: The function returns all token values as 0.

Considering the use of Claude 3 Opus, it would be beneficial to include Opus (anthropic.claude-3-opus-20240229-v1:0) in the model cost mapping. Here are the current values for reference:

langchain_community.callbacks.bedrock_anthropic_callback

MODEL_COST_PER_1K_INPUT_TOKENS = {
    "anthropic.claude-instant-v1": 0.0008,
    "anthropic.claude-v2": 0.008,
    "anthropic.claude-v2:1": 0.008,
    "anthropic.claude-3-opus-20240229-v1:0": 0.015,
    "anthropic.claude-3-sonnet-20240229-v1:0": 0.003,
    "anthropic.claude-3-haiku-20240307-v1:0": 0.00025,
}

MODEL_COST_PER_1K_OUTPUT_TOKENS = {
    "anthropic.claude-instant-v1": 0.0024,
    "anthropic.claude-v2": 0.024,
    "anthropic.claude-v2:1": 0.024,
    "anthropic.claude-3-opus-20240229-v1:0": 0.075,
    "anthropic.claude-3-sonnet-20240229-v1:0": 0.015,
    "anthropic.claude-3-haiku-20240307-v1:0": 0.00125,
}

System Info

Langchain: langchain==0.2.0rc2 langchain-anthropic==0.1.12 langchain-aws==0.1.3 langchain-chroma==0.1.0 langchain-community==0.2.0rc1 langchain-core==0.1.52 langchain-openai==0.1.6 langchain-text-splitters==0.0.1

OS: windows10

Python: 3.10.4

OSS-GR commented 1 month ago

I am facing the same issue. All token and cost values are returned as 0. Only value that displays something is the successful requests line.