microsoft / kernel-memory

RAG architecture: index and query any data using LLM and natural language, track sources, show citations, asynchronous memory patterns.
https://microsoft.github.io/kernel-memory
MIT License
1.34k stars 252 forks source link

Add Request Context feature: allow to override settings at runtime #673

Closed dluc closed 2 weeks ago

dluc commented 2 weeks ago

Motivation and Context (Why the change? What's the scenario?)

Several settings of the solution are hard coded by design, and others can be configured but require a service restart to be changed.

This PR introduces a Context object that is passed during ingestion and search/ask. The context object is optional and can contain custom key-values accessible to handlers and search clients.

During web requests, the context is accessible also via dependency injection in those scenarios where a method signature doesn't support IContext, see RequestContextProvider. During ingestion, the context is accessible through the DataPipeline instance.

The context allows to override the following settings during a request/during the upload a document, without the need to change the code or change the configuration:

Other keys can be used e.g. when working with custom handlers and custom classes. The RAG {{$facts}} template is now configurable and can include tags and metadata:

The PR includes a few examples.

Sample syntax:

var context = new RequestContext();

context.SetArg("custom_summary_prompt_str", "Summarize this: {{$input}}. Summary: ");

context.SetArg("custom_summary_overlapping_tokens_int", 0);

await memory.ImportDocumentAsync(
    new Document("doc1").AddFile("file4-KM-Readme.pdf"),
    steps: Constants.PipelineOnlySummary,
    context: context);
var context = new RequestContext();

context.SetArg("custom_rag_fact_template_str", "=== Last update: {{$meta[last_update]}} ===\n{{$content}}\n");

context.SetArg("custom_rag_prompt_str", """
                                        Facts:
                                        {{$facts}}
                                        ======
                                        Given only the timestamped facts above, provide a very short answer, include the relevant dates in brackets.
                                        If you don't have sufficient information, reply with '{{$notFound}}'.
                                        Question: {{$input}}
                                        Answer:
                                        """);

var answer = await s_memory.AskAsync("What's Kernel Memory?", context: context);