microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.14k stars 3.1k forks source link

.Net: Bug: OpenAIPromptExecutionSettings do not override kernel modelId #7072

Closed mateuszmigas closed 1 month ago

mateuszmigas commented 1 month ago

Describe the bug Function OpenAIPromptExecutionSettings do not override kernel modelId.

To Reproduce Steps to reproduce the behavior: Run the following code:

var kernel = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion("gpt-3.5-turbo", configuration["API_KEY"], null,
        null, httpClient)
    .Build();
var gpt4 = kernel.CreateFunctionFromPrompt(..., new OpenAIPromptExecutionSettings { ModelId = "gpt-4" });    
var result = await gpt4.InvokeAsync(kernel, new KernelArguments { { "input", userInput  } });
  1. Check traffic with OpenAI:
    "model": "gpt-3.5-turbo-0125",
    "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello there, greeting from GPT-4! How may I assist you today?"
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
    ]
  2. It uses kernel model instead of execution settings model.

Expected behavior It should use execution settings model.

Screenshots N/A

Platform

Additional context Code to run:


var kernel = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion("gpt-3.5-turbo", configuration["API_KEY"], null,
        null, httpClient)
    .Build();

var gpt4Def = """
             Say hi from GPT-4
             """;
var gpt4 = kernel.CreateFunctionFromPrompt(gpt4Def, new OpenAIPromptExecutionSettings { ModelId = "gpt-4" });

var gpt3TurboDef = """
                Say hi from GPT-3.5 Turbo
                """;
var gpt3Turbo = kernel.CreateFunctionFromPrompt(gpt3TurboDef, new OpenAIPromptExecutionSettings { ModelId = "gpt-3.5-turbo" });

do
{
    Console.WriteLine("Please enter prompt:");
    var userInput = Console.ReadLine();
    var result1 = await gpt3Turbo.InvokeAsync(kernel, new KernelArguments { { "input", userInput } });
    var result2 = await gpt4.InvokeAsync(kernel, new KernelArguments { { "input", userInput  } });
    Console.WriteLine(result1.GetValue<string>()!);
    Console.WriteLine(result2.GetValue<string>()!);
} while (true);
dmytrostruk commented 1 month ago

Hi @mateuszmigas , thanks for reporting this issue!

You can define different functions which will use different models, but as first step you need to register all available models in your app in Kernel, and then during function initialization or invocation you can select which model to use based on registered models in Kernel:

// Build a kernel with multiple chat completion services
var builder = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion(
        modelId: "gpt-4-0125-preview",
        apiKey: TestConfiguration.OpenAI.ApiKey)
    .AddOpenAIChatCompletion(
        modelId: "gpt-3.5-turbo-0125",
        apiKey: TestConfiguration.OpenAI.ApiKey);

Kernel kernel = builder.Build();

var prompt = "Hello AI, what can you do for me?";

var result = await kernel.InvokePromptAsync(prompt, new(new OpenAIPromptExecutionSettings { ModelId = "gpt-3.5-turbo-0125" }));
mateuszmigas commented 1 month ago

@dmytrostruk thanks for answer!