.Net: using StepwisePlanner get error from azure openai chatcomplete

[linkaiyu]

Describe the bug when using StepwisePlan in a simple console app, asure openai returns this error

Azure.RequestFailedException: 'The response was filtered due to the prompt triggering Azure OpenAI’s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766

Status: 400 (model_error)

ErrorCode: content_filter

Content:

{"error":{"message":"The response was filtered due to the prompt triggering Azure OpenAI’s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: [https://go.microsoft.com/fwlink/?linkid=2198766","type":null,"param":"prompt","code":"content_filter","status":400,"innererror":{"code":"ResponsibleAIPolicyViolation","content_filter_result":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":true,"severity":"medium"},"violence":{"filtered":false,"severity":"safe"}}}}]( https://go.microsoft.com/fwlink/?linkid=2198766","type":null,"param":"prompt","code":"content_filter","status":400,"innererror":{"code":"ResponsibleAIPolicyViolation","content_filter_result":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":true,"severity":"medium"},"violence":{"filtered":false,"severity":"safe"}}}}}

Headers:

X-Request-ID: REDACTED

ms-azureml-model-error-reason: REDACTED

ms-azureml-model-error-statuscode: REDACTED

x-ms-client-request-id: f63eec08-396f-4959-8138-8697a490f7d7

apim-request-id: REDACTED

azureml-model-session: REDACTED

azureml-model-group: REDACTED

OpenAI-Processing-Ms: REDACTED

Strict-Transport-Security: REDACTED

X-Content-Type-Options: REDACTED

x-ms-region: REDACTED

Date: Mon, 25 Sep 2023 23:31:25 GMT

Content-Length: 622

Content-Type: application/json

To Reproduce 1 create a C# console app

2. in the program.cs file, paste in this code

using Microsoft.Extensions.Logging; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.Planning; using Microsoft.SemanticKernel.SkillDefinition;

string aoaiEndpoint = Environment.GetEnvironmentVariable("AZUREOPENAI_ENDPOINT")!; string aoaiApiKey = Environment.GetEnvironmentVariable("AZUREOPENAI_API_KEY")!; string aoaiModel = "Gpt35Turbo_0301";

IKernel kernel = Kernel.Builder .WithLoggerFactory(LoggerFactory.Create(b => b.AddConsole())) .WithAzureChatCompletionService(aoaiModel, aoaiEndpoint, aoaiApiKey) .Build();

kernel.RegisterCustomFunction(SKFunction.FromNativeFunction( (string personName) => personName switch { "Jane" => 8, _ => 0 }, "Demographics", "GetAge", "Gets the age of the person whose name is provided"));

var planner = new StepwisePlanner(kernel); Plan p = planner.CreatePlan("Jane's dog is half her age. How old is the dog?"); Console.WriteLine($"Result: {await kernel.RunAsync(p)}");

3. build and run

Expected behavior the exception happens at the last line of code

Screenshots If applicable, add screenshots to help explain your problem.

Platform

• OS: [Windows]
• IDE: [Visual Studio 2022 ]
• Language: C#
• Source: [e.g. NuGet package version 0.1.0, pip package version 0.1.0, main branch of repository]

Additional context This code is copied from this devblog article in the paragraph about StevwisePlanner https://devblogs.microsoft.com/dotnet/demystifying-retrieval-augmented-generation-with-dotnet/

[matthewbolanos]

Thanks for raising; I'm going to follow up with the Azure Content Safety team to see why this is getting flagged.

[freistli]

The same issue can always be reproduced with this sample project when use gpt-35-turbo on Azure OpenAI. Seems the content filter is easily triggered with math or number plugin scenarios when use StepWisePlanner. SK version is 1.0. beta5

Client: https://github.com/freistli/semantic-kernel-docs/tree/main/samples/dotnet/14-Create-ChatGPT-Plugin/Solution

OpenAPI functions: https://github.com/freistli/semantic-kernel-docs/tree/main/samples/dotnet/14-Create-ChatGPT-Plugin/MathPlugin

[matthewbolanos]

We've since introduced a new upgraded planner that leverages Handlebars. Because of that, we plan on deprecating the current sequential planner. Can you try the same scenario with the Handlebars planner and see if the issue reproduces? If so, please create a new issue.