The get_openai_callback() function is not updated for Azure models.

[GuzmanRB]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain_openai import AzureChatOpenAI from langchain_community.callbacks import get_openai_callback from langchain_core.messages import HumanMessage, SystemMessage from langchain_core.prompts import ChatPromptTemplate

def chat_completion( temperature: int = 0): try: chat = AzureChatOpenAI( azure_deployment=MODEL, azure_endpoint=AZURE_OPENAI_ENDPOINT, api_key=AZURE_OPENAI_API_KEY, openai_api_version=API_VERSION, model_version=1106, temperature=temperature, max_tokens=MAX_TOKENS, )

        messages = [SystemMessage(content="You are a wonderful assistant"), HumanMessage(content="Write a haiku about the sea")]

        prompt = ChatPromptTemplate.from_messages(messages)
        runnable =  prompt | chat

        with get_openai_callback() as cb:
            res = runnable.invoke({})
            print( f"Total tokens: {cb.total_tokens}")
            print(f"Total cost: {cb.total_cost}")
            print(f"Haiku: {res.content}")

    except Exception as err:
        print(str(err))

Error Message and Stack Trace (if applicable)

No response

Description

The get_openai_callback() callback does not have updated models for Azure in the file: "\langchain_community\callbacks\openai_info.py"

So when making a request the total_cost returned is 0.0.

I know that the error is solved by adding the models to the dictionary: "MODEL_COST_PER_1K_TOKENS". But when deploying in docker for example it would be a tedious task to be modifying the file.

Would it be possible to add the models for Azure? https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/ Models: "gpt-35-turbo-1106": 0.001, "gpt-35-turbo-1106-completion": 0.002, "gpt-35-turbo-0125": 0.0005, "gpt-35-turbo-0125-completion": 0.0015,

System Info

langchain==0.2.6 langchain-community==0.2.6 langchain-core==0.2.10 langchain-openai==0.1.8 langchain-text-splitters==0.2.0 langsmith==0.1.82

[tibor-reiss]

They are all there: gpt-35-turbo-1106 -> gpt-3.5-turbo-1106 gpt-35-turbo-0125 -> added on 29/06/2024

[mspronesti]

The real problem here is that Azure OpenAI is not consistent with naming, e.g. gpt-35-turbo is the name of all the models of the gpt-35-turbo-* family (except the 16k, which are named gpt-35-turbo-16k), so that callback wouldn't work.