Error - benchllm with Azure OpenAi service

[FJLopezGarcia]

Hello, I am trying to use benchllm with Azure OpenAi service. I have updated the Chat example you have but I am getting this error: InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.

But sometimes the code works and perfom the evaluation properly:

below you have the code I am running. Do you know how to fix it? Thanks!!

` from dotenv import load_dotenv import os import openai import benchllm from benchllm.input_types import ChatInput

load_dotenv()

OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") OPENAI_DEPLOYMENT_ENDPOINT = os.getenv("OPENAI_DEPLOYMENT_ENDPOINT")

openai.api_type = "azure" openai.api_version = "2023-07-01-preview" openai.api_base = OPENAI_DEPLOYMENT_ENDPOINT openai.api_key = OPENAI_API_KEY

def chat(messages: ChatInput): try: response = openai.ChatCompletion.create( engine="Chat358K", messages=messages ) content = response.choices[0].message.content.strip() print("Model Response:", content) return content
except openai.OpenAIError as e: print("OpenAI Error:", e) return str(e)

@benchllm.test(suite="./tests") def gpt_3_5(input: ChatInput): print("input:", input) return chat(input) `

[FJLopezGarcia]

Do you have any update? Did you have the chance to review this issue? Thanks a lot

[Elio-Khater]

Hello, We are trying to connect BenchLLM to Azure Open Ai, and we are encountering the exact same problem that @FJLopezGarcia is facing. Is there any updates on the matter?