Bad Request on Azure Chat.

Johnz86 commented 3 weeks ago

I have working configuration, that is working with openai azure quickstart:

import os
import dotenv
from openai import AzureOpenAI

dotenv.load_dotenv()
ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT")
API_KEY = os.getenv("AZURE_OPENAI_API_KEY")

print(ENDPOINT)
print(API_KEY)

client = AzureOpenAI(
  azure_endpoint = ENDPOINT, 
  api_key=API_KEY,  
  api_version="2024-02-01"
)

response = client.chat.completions.create(
    model="project-gpt-4o-mini-2024-07-18", 
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},
        {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},
        {"role": "user", "content": "Do other Azure AI services support this too?"}
    ]
)

print(response.choices[0].message.content)

And based on that I configured similar configuration azure/config.yaml:

- model_id: gpt-4o-mini-2024-07-18
  deployment_name: project-gpt-4o-mini-2024-07-18
  endpoint: https://openai-project-sweden.openai.azure.com/
  api_version: '2024-02-01'

What I get in reponse is:

PS C:\> llm -m gpt-4o-mini-2024-07-18 "What is the capital of Sweden?"
Error: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request</h2>
<hr><p>HTTP Error 400. The request is badly formed.</p>
</BODY></HTML>

Johnz86 commented 3 weeks ago

I tried to debug the issue and found out that in:

def _get_client(self):
    return AzureOpenAI(
        api_key=self.key,
        api_version=self.api_version,
        azure_endpoint=self.endpoint,
    )

I had no self.key. For whatever reason I had no azure key set with: llm keys set azure. After I did it, the error disappeared. Since the input to key is blank, you do not see if there is anything pasted/written. Which is error prone. It would be nice to have a check for required variables.

fabge commented 3 weeks ago

feel free to submit a PR!

fabge / llm-azure

Bad Request on Azure Chat. #5