Provider of the kernel runtime

[morphy76]

The kernel binds and execute all the semantic kernel components, in particular AI services and memories.

The kernel provider needs to create a named instance so that being accordingly preconfigured for available/default AI services and memory.

Multiple kernels can be configured.

[morphy76]

I guess new issues are required to bind:

• AI services (models + client configurations)
• Memories
• Native functions
• and so on

Plz, forgive me, I'm just trying to understand the plan

[agoncal]

Yes, you're totally right @morphy76, the idea is to have this extension support memories, native functions and so on. But first I want to focus on testing (I need to understand how Semantic Kernel uses their Mock to test against OpenAI services) and documentation so we can release a first 0.1 version of this extension.

[morphy76]

Nice, I'll stay on this page, then... it should also help to create a better spec than the one I did :)

[melloware]

Teamwork!!! 😄

[morphy76]

I'm sorry to be a little bit not-present, it's a quite busy period but today I did some checks about mocking OpenAI; here it is some notes, hopefully useful:

• I gave a look to DefaultKernelTest in the semantic-kernel sources but I'm not sure about its adoption... not sure if it is feasable nor recommendable
• The OpenAI client is not easily mockable, I tried quarkiverse-mockserver, introducing an override-endpoint configuration for the openai client
  • The OpenAIClientProvider connector is not flexible enough to override the endpoint
  • Falling back to OpenAIClientBuilder (in SemanticKernelClientProducer.produceOpenAIAsyncClient) creates a wide range of configurable attributes or a wide range of default assumptions, not an issue because after all it's what OpenAIClientProvider more or less is already doing
  • Falling back to OpenAIClientBuilder also requires to provide a non-default HTTP pipeline due to a non configurable check against HTTPS, I was running mockserver using HTTP
  • Anyway, overriding the endpoint causes OpenAIClientBuilder to create an azure client, which could not be an issue in terms of testing, it's just a matter of mocked requests/responses: we're not testing openai or the quality of the answers
• The AzureOpenAI client seems to be easily mockable using quarkiverse-mockserver
  • But I guess that OpenAIClientProvider cannot be used anyway, due to my "own" issue about running mockserver using TLS

I failed to run mockserver using HTTPS due to:

• a dynamically shared CA is not accepted by the client
• I didn't verified (time was over) how to share a keystore between mockserver and the integration-tests project to hold a self-signed, shared, CA

Looking forward for hints/suggestions/ideas/recommendations/corrections/whatever; next time-slot, I'll give a better look at semantic kernel sources starting from DefaultKernelTest but I'm quite pessimistic :/ I think to push more on mockserver

(edit: workplace/lab https://github.com/morphy76/quarkus-semantic-kernel/tree/introducing_openai_server_mock)

[agoncal]

@morphy76 I know @geoand has created a LangChain4j extension and he uses OpenAI. I haven't had a look at it yet, but I remember Giorgios mentionning something about mocking OpenAI calls. I'll double check

[geoand]

We don't provide any OOTB support for mocking the calls, but it's not hard to do, and I definitely do want to have some testing utilities in soon