microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.25k stars 3.12k forks source link

New Feature: Agents story - Containerization of common frameworks (or custom code) as Plugins #7247

Open joslat opened 1 month ago

joslat commented 1 month ago

I'd like to use LLMLingua with .NEt & Java. I'd also would like to use GraphRag too And also, LlamaIndex...

What if we enable an easy way to conteinarize those SDKs or frameworks on their own container sandboxes? enabling an interface to access them from your language of choice? And this to work in your computer, in the cloud, in a build agent... maybe even also on a ACI...

A non standard approach is implemented in Kevin's beaugrand code (see https://github.com/microsoft/semantic-kernel/issues/7243) Or you can look at how we are doing at Swiss Life's Squadron here: https://github.com/SwissLife-OSS/squadron (this is for testing purposes but works flawlesssly deploying and removing containers in very short timespan (100 or less ms).

Note: This could be a generic non agent feature but I believe that the agent story will benefit TONS from it, enabling a complete decoupling from executing the code in the cloud and enabling an agent to execute much much more than was possible before. And in a simple way - extend an interface or base class and there you have. (or reference a container implementation)

joslat commented 1 month ago

Related issues: