.Net: How To Get KernelFunction Name from `IChatCompletionService.GetStreamingChatMessageContentsAsync`

[nor0x]

Describe the bug I'm developing several KernelFunctions which are invoked from a client. Is there a way to retrieve metadata from the function in the streaming responses in GetStreamingChatMessageContentsAsync from IChatCompletionService ? The function name would already help me with my usecase.

Currently the FunctionName in the response InnerContent is empty eventhough the function was called correctly:

I have already tried adding a Filter to intercept the response after the function result was created:

public class FunctionsFilter : IFunctionInvocationFilter, IAutoFunctionInvocationFilter
{
    public async Task OnAutoFunctionInvocationAsync(AutoFunctionInvocationContext context, Func<AutoFunctionInvocationContext, Task> next)
    {
        Debug.WriteLine($"Invoking {context.Function.Name}");

        await next(context);

        var result = context.Result;

        context.Result = new FunctionResult(context.Function, result.ToString(), metadata: new Dictionary<string, object>
        {
            { "function", context.Function.Name }
        });
....

there I wanted to add context.Function.Name as MetaData to the context, but as you can see on the screenshot the MetaData is not present in the response

Expected behavior IChatCompletionService.GetStreamingChatMessageContentsAsync reponse should contain information or metadata about which KernelFunctions were called

I'm using the Microsoft.SemanticKernel NuGet package version 1.17.1

[SergeyMenshykh]

Hi @nor0x, thank you for reaching out with your question.

The GetStreamingChatMessageContentsAsync, as the name implies, returns content in a streaming way - chunk by chunk. It's applicable to both text responses and function calls. If the AI model returns a text, it's up to the AI model or a hosting/streaming mechanism to decide how to chunk/generate the response - by phrase, word, or letter. Similarly, a function call may come in many chunks where the function name can be in the first chunk, and the function arguments represented by JSON can come as several following chunks. Sometimes, AI models can mix both text and function call updates in one respose.

Looking at your scenario, I've identified two solutions you can use to access function names. The first one will allow you to access the function name immediately as soon as it gets streamed - see line 282 in the example below. The other solution can be used if there's no need for immediate access to the function name - see line 287:

The third solution could be to check if the chat history has changed since the last update and, if that is the case, try to get the function call content from the new items in the chat history, like this: var functionCalls = FunctionCallContent.GetFunctionCalls(chatHistory.Last());.

Let me know if any of the suggested solutions are sufficient to implement your scenario.

[SergeyMenshykh]

Closing the issue for now. @nor0x, feel free to reopen it if neither of the solutions works for you.

[nor0x]

thank you! works just fine 🥳