Issue with AzureOpenAI gpt-4o within Langchain framework

[sam-h-long]

[ ] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug Using AzureOpenAI from langchain (from langchain_openai import AzureOpenAI) getting an error when trying to use gpt-4odeployment.

Exception raised in Job[0]: BadRequestError(Error code: 400 - {'error': {'code': 'OperationNotSupported', 'message': 'The completion operation does not work with the specified model, gpt-4o. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993.'}})

I think this originates from OpenAI Completetions API is legacy

https://platform.openai.com/docs/guides/completions

Ragas version: 0.1.16 Python version: 3.10.14

Code to Reproduce

from langchain_openai import AzureOpenAI 
from datasets import Dataset 
from ragas.metrics import context_recall
from ragas import evaluate
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

gpt_4o = AzureOpenAI(model_name="gpt-4o",
                         deployment_name="XXXXXXXXX",
                         openai_api_version="2024-06-01",
                         azure_endpoint="XXXXXXXXXXXX",
                         azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(),
                                                                           "https://cognitiveservices.azure.com/.default")
                         )

data_samples = {
    'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
    'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
    'contexts' : [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], 
    ['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']],
    'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
}
dataset = Dataset.from_dict(data_samples)
score = evaluate(dataset, metrics=[context_recall], llm=gpt_4o)
score.to_pandas()

Error trace

Exception raised in Job[0]: BadRequestError(Error code: 400 - {'error': {'code': 'OperationNotSupported', 'message': 'The completion operation does not work with the specified model, gpt-4o. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993.'}})

Expected behavior Using a gpt-35-turbo deployment works fine 😄

Additional context

[dosubot[bot]]

Hey there, @sam-h-long! I'm here to help you with any bugs, questions, or contributions you have. Let's dive in and tackle this together!

I found a similar closed issue that might help you resolve your problem. The issue was related to the AzureChatOpenAI object and had a solution that involved correcting typos and updating configurations. Here are the steps you can follow:

1. Correct the Typo in Your Code: Ensure that you are correctly assigning the AzureOpenAI instance to the llm parameter in the evaluate function. For example:

   gpt_4o = AzureOpenAI(
      model_name="gpt-4o",
      deployment_name="XXXXXXXXX",
      openai_api_version="2024-06-01",
      azure_endpoint="XXXXXXXXXXXX",
      azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
   )

2. Use the Correct Configuration Parameters: Make sure you are using the correct parameters for the AzureOpenAI model. The gpt-4o model might not support the completion operation. Instead, you should use the chat operation. Here is an example configuration:

   from langchain_openai.chat_models import AzureChatOpenAI
   
   azure_model = AzureChatOpenAI(
      openai_api_version="2023-05-15",
      azure_endpoint="https://<your-endpoint>.openai.azure.com/",
      azure_deployment="your-deployment-name",
      model="gpt-4o",
      validate_base_url=False,
   )

3. Check Supported Operations: The error message indicates that the completion operation is not supported for the gpt-4o model. You should verify the supported operations for the gpt-4o model in the Azure OpenAI documentation.

4. Example Usage: Here is an example of how to use the AzureChatOpenAI model with the evaluate function:

   from langchain_openai.chat_models import AzureChatOpenAI
   from datasets import Dataset
   from ragas.metrics import context_recall
   from ragas import evaluate
   from azure.identity import DefaultAzureCredential, get_bearer_token_provider
   
   azure_model = AzureChatOpenAI(
      openai_api_version="2023-05-15",
      azure_endpoint="https://<your-endpoint>.openai.azure.com/",
      azure_deployment="your-deployment-name",
      model="gpt-4o",
      validate_base_url=False,
   )
   
   data_samples = {
      'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
      'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
      'contexts': [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], 
                   ['The Green Bay Packers...Green Bay, Wisconsin.', 'The Packers compete...Football Conference']],
      'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
   }
   dataset = Dataset.from_dict(data_samples)
   score = evaluate(dataset, metrics=[context_recall], llm=azure_model)
   score.to_pandas()

By following these steps, you should be able to resolve the BadRequestError and correctly use the gpt-4o model with the Langchain framework ^[1][2].

To continue talking to Dosu, mention @dosu.

[sam-h-long]

Thank you @dosu

Switching from AzureOpenAI to AzureChatOpenAI worked 😄

Please close this issue

[dosubot[bot]]

You're welcome! I'm glad to hear that switching to AzureChatOpenAI resolved the issue. If you have any more questions or need further assistance, feel free to ask. Have a great day!

To continue talking to Dosu, mention @dosu.