search

microsoft / kernel-memory

RAG architecture: index and query any data using LLM and natural language, track sources, show citations, asynchronous memory patterns.
https://microsoft.github.io/kernel-memory
MIT License
1.59k stars 306 forks source link

InvalidOperationException when adding logging service #288

Closed bbence84 closed 9 months ago

bbence84 commented 9 months ago

Context / Scenario

Not sure if maybe I am doing something wrong, I am not a seasoned C# developer to be honest. :) I am trying to change the log level of KernelMemory so that I wouldn't see trace and info messages... Here is how I am initializing KernelMemory:

            var kernelMemoryBuilder = new KernelMemoryBuilder()
                .WithAzureOpenAITextEmbeddingGeneration(GetOpenAIConfig(isEmbedding: true))
                .WithAzureOpenAITextGeneration(GetOpenAIConfig())
                .WithSimpleVectorDb();
            kernelMemoryBuilder.Services.AddLogging(c => c.AddConsole().SetMinimumLevel(LogLevel.Warning));

If I don't add the last line, everything is fine. When running the program with the last line, I am getting: Unhandled exception. System.InvalidOperationException: ServiceCollectionPool contains collections of different size, and direct access by position is not allowed, to avoid inconsistent results.

What happened?

An exception is raised when calling the above.

Importance

a fix would make my life easier

Platform, Language, Versions

C#

    <PackageReference Include="Microsoft.KernelMemory.Core" Version="0.26.240121.1" />
    <PackageReference Include="Microsoft.KernelMemory.SemanticKernelPlugin" Version="0.26.240121.1" />
    <PackageReference Include="Microsoft.SemanticKernel" Version="1.1.0" />
    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Sqlite" Version="1.2.0-alpha" />
    <PackageReference Include="Microsoft.SemanticKernel.Planners.Handlebars" Version="1.2.0-preview" />
    <PackageReference Include="Spectre.Console" Version="0.48.0" />

Relevant log output

Unhandled exception. System.InvalidOperationException: ServiceCollectionPool contains collections of different size, and direct access by position is not allowed, to avoid inconsistent results.
   at Microsoft.KernelMemory.AppBuilders.ServiceCollectionPool.get_Item(Int32 index)
   at Microsoft.Extensions.DependencyInjection.Extensions.ServiceCollectionDescriptorExtensions.TryAdd(IServiceCollection collection, ServiceDescriptor descriptor)
   at Microsoft.Extensions.DependencyInjection.OptionsServiceCollectionExtensions.AddOptions(IServiceCollection services)
   at Microsoft.Extensions.DependencyInjection.LoggingServiceCollectionExtensions.AddLogging(IServiceCollection services, Action`1 configure)
   at IPBModelConfigExpert.IBPModelConfigChat.InitKernelMemoryForRAG(Kernel kernel) in C:\Users\I046471\projects\IBPModelConfigExpertAssistant\Program.cs:line 83
   at IPBModelConfigExpert.IBPModelConfigChat.Main(String[] args) in C:\Users\I046471\projects\IBPModelConfigExpertAssistant\Program.cs:line 22
   at IPBModelConfigExpert.IBPModelConfigChat.<Main>(String[] args)
bbence84 commented 9 months ago

One additional info is that actually I am only seeing trace messages upon initialization, not during retrieval, and I see them in the console output (I am not using debug console, because I am using Spectre for console formatting).

info: Microsoft.KernelMemory.Handlers.TextExtractionHandler[0]
      Handler 'extract' ready
info: Microsoft.KernelMemory.Handlers.TextPartitioningHandler[0]
      Handler 'partition' ready
info: Microsoft.KernelMemory.Handlers.SummarizationHandler[0]
      Handler 'summarize' ready
info: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Handler 'gen_embeddings' ready, 1 embedding generators
info: Microsoft.KernelMemory.Handlers.SaveRecordsHandler[0]
...
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Handler 'extract' processed pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' successfully
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Handler 'partition' processed pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' successfully
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Handler 'gen_embeddings' processed pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' successfully
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Handler 'save_records' processed pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' successfully
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' complete
dluc commented 9 months ago

Thanks for the report, this https://github.com/microsoft/kernel-memory/pull/294 will fix the problem

dluc commented 9 months ago

One additional info is that actually I am only seeing trace messages upon initialization, not during retrieval, and I see them in the console output (I am not using debug console, because I am using Spectre for console formatting).

info: Microsoft.KernelMemory.Handlers.TextExtractionHandler[0]
      Handler 'extract' ready
info: Microsoft.KernelMemory.Handlers.TextPartitioningHandler[0]
      Handler 'partition' ready
info: Microsoft.KernelMemory.Handlers.SummarizationHandler[0]
      Handler 'summarize' ready
info: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Handler 'gen_embeddings' ready, 1 embedding generators
info: Microsoft.KernelMemory.Handlers.SaveRecordsHandler[0]
...
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Handler 'extract' processed pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' successfully
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Handler 'partition' processed pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' successfully
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Handler 'gen_embeddings' processed pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' successfully
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Handler 'save_records' processed pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' successfully
info: Microsoft.KernelMemory.Pipeline.BaseOrchestrator[0]
      Pipeline 'default/22d252c61de84968bba426a0ce91286f202402020745558462504' complete

could you try configuring the level in the json configuration file?

bbence84 commented 9 months ago

Thanks for the fix! But I am now unsure how can I explicitly set the log level. This does not work anymore: kernelMemoryBuilder.Services.AddLogging(c => c.AddConsole().SetMinimumLevel(LogLevel.Warning)); I don't have a json config file in my project, I am loading config in a different way. Is Kernel Memory implicitly trying to load the config file for this?

dluc commented 9 months ago

I tested your code and the log level seems to work fine.

var kernelMemoryBuilder = new KernelMemoryBuilder()
            .WithOpenAI(openAIConfig);

        kernelMemoryBuilder.Services
            .AddLogging(c => c.AddConsole().SetMinimumLevel(LogLevel.Warning));

        var memory = kernelMemoryBuilder.Build();

        Console.WriteLine("# START");

        await memory.ImportTextAsync("In physics, mass–energy equivalence is the relationship between mass and energy " +
                                     "in a system's rest frame, where the two quantities differ only by a multiplicative " +
                                     "constant and the units of measurement. The principle is described by the physicist " +
                                     "Albert Einstein's formula: E = m*c^2", documentId: "test");

        Console.WriteLine("# END");

Output in console using LogLevel.Warning:

# START
# END

Changing LogLevel.Warning to LogLevel.Information:

# START
info: Microsoft.KernelMemory.Handlers.TextExtractionHandler[0]
      Handler 'extract' ready
info: Microsoft.KernelMemory.Handlers.TextPartitioningHandler[0]
      Handler 'partition' ready
info: Microsoft.KernelMemory.Handlers.SummarizationHandler[0]
      Handler 'summarize' ready
info: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Handler 'gen_embeddings' ready, 1 embedding generators
info: Microsoft.KernelMemory.Handlers.SaveRecordsHandler[0]
      Handler save_records ready, 1 vector storages
info: Microsoft.KernelMemory.Handlers.DeleteDocumentHandler[0]
      Handler 'private_delete_document' ready
info: Microsoft.KernelMemory.Handlers.DeleteIndexHandler[0]
      Handler 'private_delete_index' ready
info: Microsoft.KernelMemory.Handlers.DeleteGeneratedFilesHandler[0]
      Handler 'delete_generated_files' ready
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Queueing upload of 1 files for further processing [request test]
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      File uploaded: content.txt, 284 bytes
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Handler 'extract' processed pipeline 'default/test' successfully
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Handler 'partition' processed pipeline 'default/test' successfully
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Handler 'gen_embeddings' processed pipeline 'default/test' successfully
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Handler 'save_records' processed pipeline 'default/test' successfully
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Pipeline 'default/test' complete
# END

And using LogLevel.Trace:

# START
info: Microsoft.KernelMemory.Handlers.TextExtractionHandler[0]
      Handler 'extract' ready
info: Microsoft.KernelMemory.Handlers.TextPartitioningHandler[0]
      Handler 'partition' ready
info: Microsoft.KernelMemory.Handlers.SummarizationHandler[0]
      Handler 'summarize' ready
info: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Handler 'gen_embeddings' ready, 1 embedding generators
info: Microsoft.KernelMemory.Handlers.SaveRecordsHandler[0]
      Handler save_records ready, 1 vector storages
info: Microsoft.KernelMemory.Handlers.DeleteDocumentHandler[0]
      Handler 'private_delete_document' ready
info: Microsoft.KernelMemory.Handlers.DeleteIndexHandler[0]
      Handler 'private_delete_index' ready
info: Microsoft.KernelMemory.Handlers.DeleteGeneratedFilesHandler[0]
      Handler 'delete_generated_files' ready
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Queueing upload of 1 files for further processing [request test]
dbug: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Uploading 1 files, pipeline 'default/test'
dbug: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Uploading file 'content.txt', size 284 bytes
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      File uploaded: content.txt, 284 bytes
dbug: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Saving pipeline status to 'default/test/__pipeline_status.json'
dbug: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Saving pipeline status to 'default/test/__pipeline_status.json'
dbug: Microsoft.KernelMemory.Handlers.TextExtractionHandler[0]
      Extracting text, pipeline 'default/test'
dbug: Microsoft.KernelMemory.Handlers.TextExtractionHandler[0]
      Extracting text from plain text file content.txt
dbug: Microsoft.KernelMemory.Handlers.TextExtractionHandler[0]
      Saving extracted text file content.txt.extract.txt
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Handler 'extract' processed pipeline 'default/test' successfully
dbug: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Saving pipeline status to 'default/test/__pipeline_status.json'
dbug: Microsoft.KernelMemory.Handlers.TextPartitioningHandler[0]
      Partitioning text, pipeline 'default/test'
dbug: Microsoft.KernelMemory.Handlers.TextPartitioningHandler[0]
      Partitioning text file content.txt.extract.txt
dbug: Microsoft.KernelMemory.Handlers.TextPartitioningHandler[0]
      Saving 1 file partitions
dbug: Microsoft.KernelMemory.Handlers.TextPartitioningHandler[0]
      Partition size: 58 tokens
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Handler 'partition' processed pipeline 'default/test' successfully
dbug: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Saving pipeline status to 'default/test/__pipeline_status.json'
dbug: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Generating embeddings, pipeline 'default/test'
trce: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Skipping file content.txt.extract.txt (not a partition, not synthetic data)
trce: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Processing file content.txt.partition.0.txt
trce: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Generating embeddings using AI.OpenAI.OpenAITextEmbeddingGenerator, file: content.txt.partition.0.txt
dbug: Microsoft.KernelMemory.Handlers.GenerateEmbeddingsHandler[0]
      Saving embedding file content.txt.partition.0.txt.AI.OpenAI.OpenAITextEmbeddingGenerator.TODO.text_embedding
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Handler 'gen_embeddings' processed pipeline 'default/test' successfully
dbug: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Saving pipeline status to 'default/test/__pipeline_status.json'
dbug: Microsoft.KernelMemory.Handlers.SaveRecordsHandler[0]
      Saving memory records, pipeline 'default/test'
trce: Microsoft.KernelMemory.Handlers.SaveRecordsHandler[0]
      Creating index 'default'
trce: Microsoft.KernelMemory.Handlers.SaveRecordsHandler[0]
      Saving record d=test//p=875303feb8494f12b8b9a71c53c47325 in index 'default'
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Handler 'save_records' processed pipeline 'default/test' successfully
dbug: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Saving pipeline status to 'default/test/__pipeline_status.json'
info: Microsoft.KernelMemory.Pipeline.InProcessPipelineOrchestrator[0]
      Pipeline 'default/test' complete
# END
dluc commented 9 months ago

For reference, code here: https://github.com/microsoft/kernel-memory/blob/main/examples/206-dotnet-configuration-and-logging/Program.cs

bbence84 commented 9 months ago

I tested your code and the log level seems to work fine.

Thanks! I really don't know what I am missing, as I am getting the below error message:

...\plugins\DocuGroundingRAGPlugin.cs(95,42): error CS1061: 'ServiceCollectionPool' does not contain a definition for 'AddLogging' and no accessible extension method 'AddLogging' accepting a first argument of type 'ServiceCollectionPool' could be found (are you missing a using directive or an assembly reference?) [...\ExpertAssistant.csp 
roj]

Initialization:

            var kernelMemoryBuilder = new KernelMemoryBuilder()
                .WithAzureOpenAITextEmbeddingGeneration(GetOpenAIConfig(isEmbedding: true))
                .WithAzureOpenAITextGeneration(GetOpenAIConfig())
                .WithSimpleVectorDb(new SimpleVectorDbConfig { StorageType = FileSystemTypes.Disk, Directory = "vector_storage"});
            kernelMemoryBuilder.Services.AddLogging(c => c.AddConsole().SetMinimumLevel(LogLevel.Warning)); 

            kernelMemory = kernelMemoryBuilder.Build();

Imports:

using System.ComponentModel;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.KernelMemory;
using Microsoft.KernelMemory.FileSystem.DevTools;
using Microsoft.KernelMemory.MemoryStorage.DevTools;
using Spectre.Console;

Package versions:


    <PackageReference Include="Microsoft.KernelMemory.Core" Version="0.27.240207.1" />
    <PackageReference Include="Microsoft.KernelMemory.SemanticKernelPlugin" Version="0.27.240207.1" />
    <PackageReference Include="Microsoft.SemanticKernel" Version="1.3.0" />
    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Sqlite" Version="1.3.0-alpha" />
    <PackageReference Include="Microsoft.SemanticKernel.Planners.Handlebars" Version="1.3.0-preview" />
    <PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
    <PackageReference Include="RestSharp" Version="110.2.0" />
    <PackageReference Include="Spectre.Console" Version="0.48.0" />
    <PackageReference Include="Spectre.Console.Cli" Version="0.48.0" />

(I just updated to the version from a few hours before, but even before that it was not working).

dluc commented 9 months ago

Could you try this code https://github.com/microsoft/kernel-memory/blob/main/examples/206-dotnet-configuration-and-logging/Program.cs ?

bbence84 commented 9 months ago

Could you try this code https://github.com/microsoft/kernel-memory/blob/main/examples/206-dotnet-configuration-and-logging/Program.cs ?

Thanks, it works now. I had to add the following references: using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging;