microsoft / kernel-memory

RAG architecture: index and query any data using LLM and natural language, track sources, show citations, asynchronous memory patterns.
https://microsoft.github.io/kernel-memory
MIT License
1.52k stars 293 forks source link

[Bug] evaluation Unable to execute normally #655

Closed xuzeyu91 closed 3 months ago

xuzeyu91 commented 3 months ago

Context / Scenario

I am working on a RAG project

What happened?

System.InvalidOperationException:“No service for type 'Microsoft.KernelMemory.Evaluation.TestSetGenerator' has been registered.”

https://github.com/microsoft/kernel-memory/pull/519 It can run normally before this PR And now I have reported the error above

using Microsoft.KernelMemory;
using Microsoft.KernelMemory.AI.OpenAI;
using Microsoft.KernelMemory.Evaluation;
using Microsoft.SemanticKernel;

var azureOpenAITextConfig = new AzureOpenAIConfig()
{
    APIKey = "xxx",
    Endpoint = "https://xxx.openai.azure.com/",
    Deployment = "gpt4-turbo",
    Auth = AzureOpenAIConfig.AuthTypes.APIKey,
    APIType = AzureOpenAIConfig.APITypes.TextCompletion,
};
var azureOpenAIEmbeddingConfig = new AzureOpenAIConfig() 
{
    APIKey = "xxx",
    Endpoint = "https://xxx.openai.azure.com/",
    Deployment = "text-embedding-ada-002",
    Auth = AzureOpenAIConfig.AuthTypes.APIKey,
    APIType = AzureOpenAIConfig.APITypes.EmbeddingGeneration,
};

var memoryBuilder = new KernelMemoryBuilder()
    .WithAzureOpenAITextGeneration(azureOpenAITextConfig)
    .WithAzureOpenAITextGeneration(azureOpenAIEmbeddingConfig)
    ;

var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion("gpt4-turbo", "https://xxx.openai.azure.com/", "xxx"
    ).Build();

var testSetGenerator = new TestSetGeneratorBuilder(memoryBuilder.Services)
                            .AddEvaluatorKernel(kernel)
                            .Build();

var distribution = new Distribution
{
    Simple = .5f,
    Reasoning = .16f,
    MultiContext = .17f,
    Conditioning = .17f
};

var testSet = testSetGenerator.GenerateTestSetsAsync(index: "default", count: 10, retryCount: 3, distribution: distribution);

I have reviewed the code for this PR and found it in the Return this. _serviceCollection. BuildServiceProvider() . GetRequiredService(); There is an error here, I did not see how TestSetGenerator was injected into the _serviceCollection

Importance

a fix would make my life easier

Platform, Language, Versions

The latest version

Relevant log output

No response

xuzeyu91 commented 3 months ago

This error has not been resolved,Have you tested this code and it can run smoothly? @kbeaugrand

xuzeyu91 commented 3 months ago
System.InvalidOperationException:“A suitable constructor for type 'Microsoft.KernelMemory.Evaluation.TestSetGenerator' could not be located. Ensure the type is concrete and services are registered for all parameters of a public constructor.”
kbeaugrand commented 3 months ago

my bad @xuzeyu91. This application is still in progress and the code remains in really early stage. Sorry for the missing and the bugs.

I provided a new update with functional tests at #657.

xuzeyu91 commented 3 months ago

Thank you very much. It can now run normally