Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.23k stars 4.57k forks source link

How to call an OpenAI service which is behind an API Management instance? #44995

Open OfirD1 opened 1 month ago

OfirD1 commented 1 month ago

Library name and version

Azure.AI.OpenAI 2.0.0-beta.2

Query/Question

I'm trying to call the chat completion endpoint of a gpt-4o deployment which is behind an API Management instance - but can't. It seems like either the url gets manipulated to an incorrect format, or the subscription key is not used as it should.

Trial no. 1 - Using OpenAIClient

In my first trial, I generally (not fully, because the Nuget version is different) followed the example appearing in the AzureOpenAI-with-APIM documentation:

Code:

// apim_url is taken from my apim overview page 
var apim_url = "redacted";

// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";

var url = $"{apim_url}/deployments/gpt-4o/chat/completions?api-version=2024-02-01";

var openAIClient = new OpenAIClient(
    credential: new System.ClientModel.ApiKeyCredential(subscription_key),
    options: new OpenAIClientOptions() { Endpoint = new Uri(url) }
);
var chatClient = openAIClient.GetChatClient("gpt-4o");
var completion = chatClient.CompleteChat(new ChatMessage[] {
    new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
    new UserChatMessage("Hi, can you help me?"),
});

Expected behavior CompleteChat should be successful.

Actual behavior 404 error.

Possible Failure Reason

Looking at my APIM logs, I see that the actual url used was wrong - chat/completion seems to have been duplicated by the client:

image

Trial no. 2 - Again using OpenAIClient, but with a different url

Looking at the above log screenshot, I figured that if I'd change the url I'd get the correct form, which I did! But apparently the subscription key is not used.

Code:

// apim_url is taken from my apim overview page 
var apim_url = "redacted";

// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";

var url = $"{apim_url}/deployments/gpt-4o?api-version=2024-02-01";

var openAIClient = new OpenAIClient(
    credential: new System.ClientModel.ApiKeyCredential(subscription_key),
    options: new OpenAIClientOptions() { Endpoint = new Uri(url) }
);
var chatClient = openAIClient.GetChatClient("gpt-4o");
var completion = chatClient.CompleteChat(new ChatMessage[] {
    new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
    new UserChatMessage("Hi, can you help me?"),
});

Expected behavior CompleteChat should be successful.

Actual behavior 401 error (not 404!)

Possible Failure Reason

Looking at my APIM logs, I see that the actual url used was now correct - but apparently the subscription key wasn't used:

image

Trial no. 3 - Using AzureOpenAIClient

After the previous two unsuccessful attempts, I tried using the AzureOpenAIClient, but I just got an even more malformed url.

Code:

// apim_url is taken from my apim overview page 
var apim_url = "redacted";

// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";

var url = $"{apim_url}/deployments/gpt-4o/chat/completions?api-version=2024-02-01";

var azureOpenAiClient = new AzureOpenAIClient(
    endpoint: new Uri(url), 
    credential: new ApiKeyCredential(subscription_key)
);
var chatClient = azureOpenAiClient.GetChatClient("gpt-4o");
var completion = chatClient.CompleteChat(new ChatMessage[] {
    new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
    new UserChatMessage("Hi, can you help me?"),
});

Expected behavior CompleteChat should be successful.

Actual behavior 404 error.

Possible Failure Reason Looking at my APIM logs, I see that the actual url used was wrong again - both deployments/gpt-4o/chat/completions and api-version seems to have been duplicated by the client:

image

Question
How can I make it work?

Environment

.NET SDK: Version: 8.0.303

Runtime Environment: OS Name: Windows OS Version: 10.0.22631

IDE: Microsoft Visual Studio Enterprise 2022 (64-bit) - Version 17.6.5

github-actions[bot] commented 1 month ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @ralph-msft @trrwilson.