Open TaoChenOSU opened 3 months ago
In the above code WriterPlugin.ShortPoem
is called directly but also providing it as a function to the model for it to invoke and expecting it to call it. The ShortPoem
prompt starts with Generate a short funny poem
and the input also has Write a poem about John Doe
. When the model is called it doesn't know it's already being called from WriterPlugin.ShortPoem
, it sees that function advertised and it's a good fit for the ask so it cals it again.
There are a few things that could help here:
AutoInvokeKernelFunctions
as it's not needed because the required function is being called directlyvar poem = await kernel.InvokePromptAsync
with the input and allow the model decide what function(s) to call // Copyright (c) Microsoft. All rights reserved.
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Resources;
namespace GettingStarted;
/// <summary>
/// This example shows how to create a prompt <see cref="KernelFunction"/> from a YAML resource.
/// </summary>
public sealed class Generate_Poem(ITestOutputHelper output) : BaseTest(output)
{
/// <summary>
/// Show how to create a prompt <see cref="KernelFunction"/> from a YAML resource.
/// </summary>
[Fact]
public async Task RunAsync()
{
// Create a kernel with OpenAI chat completion
Kernel kernel = Kernel.CreateBuilder()
.AddOpenAIChatCompletion(
modelId: TestConfiguration.OpenAI.ChatModelId,
apiKey: TestConfiguration.OpenAI.ApiKey)
.Build();
kernel.FunctionInvocationFilters.Add(new FunctionInvocationFilter(Output));
// Load prompt from resource
var generatePoemYaml = EmbeddedResource.Read("GeneratePoem.yaml");
var function = kernel.CreateFunctionFromPromptYaml(generatePoemYaml);
// Add to the Kernel
kernel.Plugins.AddFromFunctions("WriterPlugin", [function]);
// Invoke the prompt function and display the result
OpenAIPromptExecutionSettings settings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };
// Don't use AutoInvokeKernelFunctions
Console.WriteLine(await kernel.InvokeAsync(function, arguments: new()
{
{ "input", "Write a poem about John Doe" },
}));
Console.WriteLine();
// Call prompt with AutoInvokeKernelFunctions
Console.WriteLine(await kernel.InvokePromptAsync("Write a poem about John Doe", arguments: new(settings)));
}
private sealed class FunctionInvocationFilter(ITestOutputHelper output) : IFunctionInvocationFilter
{
private readonly ITestOutputHelper _output = output;
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
{
this._output.WriteLine($"FunctionInvoking - {context.Function.PluginName}.{context.Function.Name}");
await next(context);
this._output.WriteLine($"FunctionInvoked - {context.Function.PluginName}.{context.Function.Name}");
}
}
}
name: GeneratePoem
template: |
Generate a short funny poem or limerick to explain the given event. Be creative and be funny. Let your imagination run wild.
Event:{{$input}}
template_format: semantic-kernel
description: Turn a scenario into a short and entertaining poem.
input_variables:
- name: input
description: The scenario to turn into a poem.
is_required: true
output_variable:
description: The generated poem.
execution_settings:
default:
max_tokens: 60
temperature: 0.5
top_p: 0.0
presence_penalty: 0.0
frequency_penalty: 0.0
The sample just shows how to reproduce the issue. In reality, the execution settings with auto invoke enabled could come from somewhere else, such as a service selector. It's fairly easy for someone to have auto invoke enabled by default in their apps and still invoke registered functions using InvokeAsync
, which could silently result in multiple round trips to the model.
I suggest we have a list of anti-patterns somewhere if we don't suggest users to have auto invoke on and still plan to invoke registered functions directly.
Say someone has set up a service selector that returns some default settings with auto invoke enabled.
bool TrySelectAIService<T>(
Kernel kernel,
KernelFunction function,
KernelArguments arguments,
[NotNullWhen(true)] out T? service,
out PromptExecutionSettings? serviceSettings) where T : class, IAIService;
And in their apps, they invoke functions (because nothing suggests they should not). They wouldn't notice anything different but in reality, they could be consuming more tokens they need, and their apps will be slower.
Agreed. Possibly we shouldn't advertise the current function to the model, I'm not sure if that would ever be useful.
We have a task to provide more control over what kernel functions are advertised to the model, I can include this use case as part of that.
@TaoChenOSU @markwallace-microsoft is this issue still needed?
@TaoChenOSU @markwallace-microsoft is this issue still needed?
Are we still advertising the current function to the model? If so, we still have the issue.
Describe the bug When a kernel function is invoked directly via
, and with auto invoke enabled, it may result in multiple calls to the function, which means multiple calls to the model, causing unnecessary round trips.
To Reproduce Steps to reproduce the behavior:
WriterPlugin
to the project root.public sealed class Program { ///
/// The main entry point for the application.
///
/// A representing the asynchronous operation.
public static async Task Main()
{
var loggerFactory = new NullLoggerFactory();
var kernel = GetKernel(loggerFactory);
pragma warning disable SKEXP0001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
pragma warning restore SKEXP0001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
pragma warning disable SKEXP0001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
pragma warning restore SKEXP0001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
}