Using Semantic Kernel to call OpenAI which is behind an API Management instance

[OfirD1]

How can I use Semantic Kernel to call an OpenAI chat completion (gpt-4o, in my case) which is behind an API Management instance?

If such usage of Semantic Kernel is possible, I'd really like a code snippet to help with this specific use case (I've experienced with other use cases). I found related issues (#2744 and #5763), but their included links don't work anymore.

To be honest, I'm not even sure what endpoint url and api key should I use. For the time being, I tried the url (redacted) marked in the image, and the api key of the OpenAI service itself, but it doesn't work.

[dmytrostruk]

@OfirD1 Based on this documentation, you should use https://INSERT_APIM_URL_HERE/deployments/INSERT_DEPLOYMENT_NAME_HERE/chat/completions?api-version=INSERT_API_VERSION_HERE as Endpoint and APIM Subscription Key as API key.

It should be possible to use one of the AddAzureOpenAIChatCompletion methods to pass this information. If it doesn't work for you, you can create instance of OpenAIClient on your side, configure it based on your environment and pass it to Semantic Kernel. Here is an example of custom OpenAIClient, which was also mentioned in one of the discussions you shared above (the sample was moved to another location recently): https://github.com/microsoft/semantic-kernel/blob/32d3f5d01db422ccc7598c074b836bdfef94b26c/dotnet/samples/Concepts/ChatCompletion/OpenAI_CustomAzureOpenAIClient.cs#L28-L40

Please let us know if this resolves your issue. Thanks!

[OfirD1]

@dmytrostruk thanks, unfortunately it doesn't quite work. Here's what I see:

• Note: in the following, any mention of {apim_url} is a reduction of my actual api management gateway url.

1. I tried the python example given in the documentation you linked to (which simply uses the requests package) - and it worked (this is just a sanity test; I do need to use Semantic Kernel in my case).

   • Url used: {apim_url}/deployments/gpt-4o/chat/completions?api-version=2024-02-01
   • Url logged (I can see it using ApiManagementGatewayLogs on the APIM end): same as url used.

2. I then tried the Semantic Kernel builder usage you quoted from that documentation in your snippet - that doesn't work. As it seems, the url is being manipulated by Semantic Kernel into an incorrect format.

   • Url used: same as in 1.
   • Url logged: {apim_url}/deployments/gpt-4o/chat/completions/deployments/gpt-4o/chat/completions?api-version=2024-02-01&/openai=&api-version=2024-04-01-preview

3. Somewhat off-topic, but I then also tried using directly the OpenAIClient's GetChatCompletionsAsync which is also mentioned in the documentation you linked, and it also doesn't work.

   • Url used: same as in 1.
   • Url logged: same as in 2.

Screenshots

• Screenshot from my APIM logs:

• Just to reassure that I indeed use the correct url on my OpenAIClient, I'm attaching a screenshot of my url variable as seen in my debug Watch window (A watch over the client itself is also here, but for some reason the _endpoint field refuses to display):

• To clarify why do I use that certain api version of 2024-02-01 - it's because that's the latest (non-preview) available api version when setting up the Azure OpenAI api service. In any case, the 2024-04-01-preview value - which is apparently used by the OpenAiClient - is not an available option:

[nehalakhani]

Hello @dmytrostruk - Can you please help with Python example on how to access Semantic Kernel with APIM URL. I am adding below parameters to AzureChatCompletion but getting error - Access denied due to missing subscription key. Is there any other parameter available to mention Ocp-Apim-SubscriptionKey

from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion azure_chat_service= AzureChatCompletion( api_key=Ocp-Apim-SubscriptionKey endpoint={apim_url}/deployments/gpt-4o/chat/completions/deployments/gpt-4o/chat/completions?api-version=2024-02-15-preview deployment_name='gpt-4o' api_version='2024-02-15-preview' )

[nehalakhani]

This has been resolved now in this thread - https://github.com/microsoft/semantic-kernel/discussions/8340