Open nmoeller opened 2 weeks ago
moonbox3
commented
2 weeks ago Thanks for filing the feature request, @nmoeller. Could this be accomplished with our prompt filter? Not everyone may want the added latency of every request going through a content safety pipeline (I don't have data to back this up, but just a thought). We will discuss as a team, again thanks!
nmoeller
commented
2 weeks ago @moonbox3 just to clarify this is something that not everybody has to use, the simply idea would be to have something like kernel.add_content_safety(ContentSafetyConnectorBase()) and then the kernel check prompts & responses automatically. If the user does not add the ContentSafety to the Kernel everything will work like usual.
I think something like this is available in the dotnet already.
But i will also check if this can be archived via prompt filters !
moonbox3
commented
2 weeks ago Understood. I see that in .Net they are using the IPromptRenderFilter interface to create the dependency for something like TextModerationFilter. I see in their getting started docs, one can download the appropriate client. This could be included in a function that implements our prompt filter and then makes the call to the content safety service.
Name : Content Safety Layer for Python
Abstract:
The idea is to have some kind of Interceptor when invoking API's that checks for Content Safety, to ensure a stable Content Safety when switching the underlaying Model. Also with Embedded Content Safety Containers, flows
Why is this a good Idea ?
When People switching from GPT4 to Anthropic for Example, they would loose the Content Safety Feature of the Azure AI Services. To ensure Responsible AI when switch between Models, we should have a layer in the Kernel to keep Content Safety when switching Model consistent.
Also when using Hugging Face Models or Onnx Models locally there is no support for content safety except in the model itself. To enable all kind of Models to work with Content Safety a seperate layer in the Kernel would be beneficial.
As far as i saw, this feature is already available in the C# Version of semantic kernel.
How can this be implemented ?
The idea here would be to add a new abstract class named
ContentSafetyConnectorBase, this class would have abstract methods to enforce subclasses to implement them. Before and After sending data to the Models we could intercept the questions/answers in theChatCompletionClientBase. We could also add anContentSafetyExceptionin the Semantic Kernel Exceptions and the user could catch theContentSafetyExceptionand deal with it accordingly.We could also add the
AzureContentSafetyService as a Connector for samples. Also there is a offline container of Content Safety, so we could have a complete offline example with SLM's and Content Safety.