Closed yuichiromukaiyama closed 1 week ago
I apologize for including multiple topics. Additionally, is there documentation on which API versions (Azure OpenAI) are supported by Semantic Kernel?
Hi @yuichiromukaiyama, I see a few things here.
Firstly, I noticed a comment in your code around getting the execution settings. Instead of having to perform two casts, you can do something like:
kernel.get_service(service_id).get_prompt_execution_settings_class()(service_id=service_id)
or
req_settings = kernel.get_prompt_execution_settings_from_service_id(service_id=service_id)
You'll get the proper type back for the specified service_id. And you can provide the type hint like:
req_settings: AzureChatPromptExecutionSettings = kernel.get_prompt_execution_settings_from_service_id(...)
. Or even do an assert after to satisfy that it is of the expected/correct type.
Now onto your error: how is content filter=Enabled Asynchronous Filter
getting configured? I am not seeing it as part of the code you show above.
I see that a community member created this PR (#7115) that is related to the issue (the code change then caused many tests to fail and there hasn't been follow up yet). I do want to bring this up to the AzureOpenAI team to make sure this is intended and it isn't a regression. Allow me to do so and I will respond here when I hear from them.
req_settings = kernel.get_prompt_execution_settings_from_service_id(service_id=service_id)
Thank you! It worked well.
Now onto your error: how is content filter=Enabled Asynchronous Filter getting configured? I am not seeing it as part of the code you show above.
This is not something specified in the code. It is a setting on the Azure Portal side. https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new#asynchronous-filter
The settings are configured through the web portal as follows. Azure OpenAI usually returns multiple chunks together. This setting is enabled to allow the chunks to be returned more in real-time.
I see that a community member created this PR (https://github.com/microsoft/semantic-kernel/pull/7115) that is related to the issue (the code change then caused many tests to fail and there hasn't been follow up yet). I do want to bring this up to the AzureOpenAI team to make sure this is intended and it isn't a regression. Allow me to do so and I will respond here when I hear from them.
Got it. Thank you!
Hi @yuichiromukaiyama, I appreciate your response. I have reached out to a contact on the Azure OpenAI side, and they will be back in the office early next week. Once they investigate, I will respond. Thanks for your patience.
merge
pull request https://github.com/microsoft/semantic-kernel/pull/8075
Describe the bug Azure OpenAI has a content filter. The traditional Azure content filter buffers chunks for a certain period, causing a bit of a sluggish behavior compared to the OpenAI API. Recently, an Asynchronous Filter was released to enable smoother returns. However, when this feature is enabled, the following error occurs.
To Reproduce The steps to reproduce the issue are as follows: use the code below and enable the Asynchronous Filter on the Azure OpenAI side. Note that this error does not occur with api_version 2023-03-15-preview.
semantic-kernel Version: 1.1.2 model=Azure OpenAI japan-east gpt-35-turbo-16k version-0613 content filter=Enabled Asynchronous Filter api_version=2024-05-01-preview,2024-02-01,2023-12-01-preview,2023-09-01-preview,2023-08-01-preview,2023-07-01-preview
Expected behavior I expect the stream to run to completion without this error occurring.
Screenshots none
Platform
Additional context
Comparing the streams obtained from Azure OpenAI, the last chunk differs as follows.
content filter = OFF
content filter = ON
Would it be possible to include cases where None is passed as an argument here?
semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py