microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.93k stars 3.27k forks source link

Python: New Feature: Content Safety Layer for Python #9536

Open nmoeller opened 1 week ago

nmoeller commented 1 week ago

Name : Content Safety Layer for Python

Abstract:

The idea is to have some kind of Interceptor when invoking API's that checks for Content Safety, to ensure a stable Content Safety when switching the underlaying Model. Also with Embedded Content Safety Containers, flows

Why is this a good Idea ?

When People switching from GPT4 to Anthropic for Example, they would loose the Content Safety Feature of the Azure AI Services. To ensure Responsible AI when switch between Models, we should have a layer in the Kernel to keep Content Safety when switching Model consistent.

Also when using Hugging Face Models or Onnx Models locally there is no support for content safety except in the model itself. To enable all kind of Models to work with Content Safety a seperate layer in the Kernel would be beneficial.

As far as i saw, this feature is already available in the C# Version of semantic kernel.

How can this be implemented ?

The idea here would be to add a new abstract class named ContentSafetyConnectorBase, this class would have abstract methods to enforce subclasses to implement them. Before and After sending data to the Models we could intercept the questions/answers in the ChatCompletionClientBase. We could also add an ContentSafetyException in the Semantic Kernel Exceptions and the user could catch the ContentSafetyException and deal with it accordingly.

We could also add the AzureContentSafety Service as a Connector for samples. Also there is a offline container of Content Safety, so we could have a complete offline example with SLM's and Content Safety.


moonbox3 commented 1 week ago

Thanks for filing the feature request, @nmoeller. Could this be accomplished with our prompt filter? Not everyone may want the added latency of every request going through a content safety pipeline (I don't have data to back this up, but just a thought). We will discuss as a team, again thanks!

nmoeller commented 1 week ago

@moonbox3 just to clarify this is something that not everybody has to use, the simply idea would be to have something like kernel.add_content_safety(ContentSafetyConnectorBase()) and then the kernel check prompts & responses automatically. If the user does not add the ContentSafety to the Kernel everything will work like usual.

I think something like this is available in the dotnet already.

https://github.com/microsoft/semantic-kernel/tree/711afeae05eeea81e404b600d30bca2bfd206d0c/dotnet/samples/Demos/ContentSafety

But i will also check if this can be archived via prompt filters !

moonbox3 commented 1 week ago

Understood. I see that in .Net they are using the IPromptRenderFilter interface to create the dependency for something like TextModerationFilter. I see in their getting started docs, one can download the appropriate client. This could be included in a function that implements our prompt filter and then makes the call to the content safety service.

https://learn.microsoft.com/en-us/azure/ai-services/content-safety/quickstart-text?tabs=visual-studio%2Cwindows&pivots=programming-language-python