.Net: Bug: Cannot execute KernelFunctions when using - phi-3 cpu-int4-rtn-block-32

[Vizeo]

Cannot execute KernelFunctions when using - phi-3 cpu-int4-rtn-block-32.

Running the following code should return "Ironman". But the function never gets called.

To Reproduce Run Code using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.ChatCompletion; using Microsoft.SemanticKernel.Connectors.OpenAI; using System.ComponentModel;

namespace AI_test { internal class Program { public class SuperHeroPlugin { [KernelFunction, Description("The best super hero")] public string TheBestSuperHero() => "Ironman"; }

    static async Task Main(string[] args)
    {
        // Your PHI-3 model location 
        var modelPath = @"D:\AI Models\Phi-3\cpu_and_mobile\cpu-int4-rtn-block-32";

        // Load the model and services
        var builder = Kernel.CreateBuilder();
        builder.AddOnnxRuntimeGenAIChatCompletion("phi-3", modelPath);
        builder.Services.AddLogging(c => c.AddConsole().SetMinimumLevel(LogLevel.Trace));
        builder.Plugins.AddFromType<SuperHeroPlugin>();
        var kernel = builder.Build();

        // Create services such as chatCompletionService and embeddingGeneration
        var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();
        ChatHistory chat = new();
        OpenAIPromptExecutionSettings settings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };

        // Start the conversation
        Console.Write("Who is the best superhero?");
        var question = "Who is the best superhero?";

        ChatMessageContent chatResult = await chatCompletionService.GetChatMessageContentAsync(question, settings, kernel);

        Console.Write($"\n>>> Result: {chatResult}\n\n> ");
    }
}

Expected behavior returns "Ironman"

Platform

• OS: Windows
• IDE: Visual Studio 2022
• Language: C#

[markwallace-microsoft]

@Vizeo This is currently not supported by ONNX GenAI, these are the tracking issues

• https://github.com/microsoft/onnxruntime-genai/discussions/969
• https://github.com/microsoft/semantic-kernel/issues/8096