This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.23k
stars
4.57k
forks
source link
How to call an OpenAI service which is behind an API Management instance? #44995
I'm trying to call the chat completion endpoint of a gpt-4o deployment which is behind an API Management instance - but can't.
It seems like either the url gets manipulated to an incorrect format, or the subscription key is not used as it should.
Trial no. 1 - Using OpenAIClient
In my first trial, I generally (not fully, because the Nuget version is different) followed the example appearing in the AzureOpenAI-with-APIM documentation:
Code:
// apim_url is taken from my apim overview page
var apim_url = "redacted";
// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";
var url = $"{apim_url}/deployments/gpt-4o/chat/completions?api-version=2024-02-01";
var openAIClient = new OpenAIClient(
credential: new System.ClientModel.ApiKeyCredential(subscription_key),
options: new OpenAIClientOptions() { Endpoint = new Uri(url) }
);
var chatClient = openAIClient.GetChatClient("gpt-4o");
var completion = chatClient.CompleteChat(new ChatMessage[] {
new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
new UserChatMessage("Hi, can you help me?"),
});
Expected behaviorCompleteChat should be successful.
Actual behavior
404 error.
Possible Failure Reason
Looking at my APIM logs, I see that the actual url used was wrong - chat/completion seems to have been duplicated by the client:
Trial no. 2 - Again using OpenAIClient, but with a different url
Looking at the above log screenshot, I figured that if I'd change the url I'd get the correct form, which I did! But apparently the subscription key is not used.
Code:
// apim_url is taken from my apim overview page
var apim_url = "redacted";
// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";
var url = $"{apim_url}/deployments/gpt-4o?api-version=2024-02-01";
var openAIClient = new OpenAIClient(
credential: new System.ClientModel.ApiKeyCredential(subscription_key),
options: new OpenAIClientOptions() { Endpoint = new Uri(url) }
);
var chatClient = openAIClient.GetChatClient("gpt-4o");
var completion = chatClient.CompleteChat(new ChatMessage[] {
new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
new UserChatMessage("Hi, can you help me?"),
});
Expected behaviorCompleteChat should be successful.
Actual behavior
401 error (not 404!)
Possible Failure Reason
Looking at my APIM logs, I see that the actual url used was now correct - but apparently the subscription key wasn't used:
Trial no. 3 - Using AzureOpenAIClient
After the previous two unsuccessful attempts, I tried using the AzureOpenAIClient, but I just got an even more malformed url.
Code:
// apim_url is taken from my apim overview page
var apim_url = "redacted";
// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";
var url = $"{apim_url}/deployments/gpt-4o/chat/completions?api-version=2024-02-01";
var azureOpenAiClient = new AzureOpenAIClient(
endpoint: new Uri(url),
credential: new ApiKeyCredential(subscription_key)
);
var chatClient = azureOpenAiClient.GetChatClient("gpt-4o");
var completion = chatClient.CompleteChat(new ChatMessage[] {
new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
new UserChatMessage("Hi, can you help me?"),
});
Expected behaviorCompleteChat should be successful.
Actual behavior
404 error.
Possible Failure Reason
Looking at my APIM logs, I see that the actual url used was wrong again - both deployments/gpt-4o/chat/completions and api-version seems to have been duplicated by the client:
Question
How can I make it work?
Environment
.NET SDK:
Version: 8.0.303
Runtime Environment:
OS Name: Windows
OS Version: 10.0.22631
IDE:
Microsoft Visual Studio Enterprise 2022 (64-bit) - Version 17.6.5
Library name and version
Azure.AI.OpenAI 2.0.0-beta.2
Query/Question
I'm trying to call the chat completion endpoint of a
gpt-4o
deployment which is behind an API Management instance - but can't. It seems like either the url gets manipulated to an incorrect format, or the subscription key is not used as it should.Trial no. 1 - Using
OpenAIClient
In my first trial, I generally (not fully, because the Nuget version is different) followed the example appearing in the AzureOpenAI-with-APIM documentation:
Code:
Expected behavior
CompleteChat
should be successful.Actual behavior 404 error.
Possible Failure Reason
Looking at my APIM logs, I see that the actual url used was wrong -
chat/completion
seems to have been duplicated by the client:Trial no. 2 - Again using
OpenAIClient
, but with a different urlLooking at the above log screenshot, I figured that if I'd change the url I'd get the correct form, which I did! But apparently the subscription key is not used.
Code:
Expected behavior
CompleteChat
should be successful.Actual behavior 401 error (not 404!)
Possible Failure Reason
Looking at my APIM logs, I see that the actual url used was now correct - but apparently the subscription key wasn't used:
Trial no. 3 - Using
AzureOpenAIClient
After the previous two unsuccessful attempts, I tried using the
AzureOpenAIClient
, but I just got an even more malformed url.Code:
Expected behavior
CompleteChat
should be successful.Actual behavior 404 error.
Possible Failure Reason Looking at my APIM logs, I see that the actual url used was wrong again - both
deployments/gpt-4o/chat/completions
andapi-version
seems to have been duplicated by the client:Question
How can I make it work?
Environment
.NET SDK: Version: 8.0.303
Runtime Environment: OS Name: Windows OS Version: 10.0.22631
IDE: Microsoft Visual Studio Enterprise 2022 (64-bit) - Version 17.6.5