search

microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.72k stars 3.22k forks source link

Integration of Semantic Kernel and AzureBotFramework SDK #4848

Open nmoeller opened 8 months ago

nmoeller commented 8 months ago

The big Picture in my Mind

GenAI Applications should be capable of executing every workload to succeed in the market. Currently I see GenAI Applications more as something that works in a passive way (retrieving data / answering questions), but the final idea is they should be able to execute critical workload aswell to really impact the industry and change the market.

The Problem

When extracting Parameters from Prompts to Function Calling they can run into hallucinations. To ensure to execute the workload a human check would be necessary. This check should be to 100% deterministic and not involve a LLM to ensure the workload is executed as desired.

The Solution i see

The collaboration between Semantic Kernel and the BotFramework could be key here to succeed in the market. The overall idea is to use Planners to run the corresponding function's and then when a critical function is called you use the Hooks that are available on Semantic Kernel to start a Dialog in BotFramework SDK to get into the "deterministic Mode" to ensure the workload is executed with the correct parameters.

This is probably already possible when we start a Dialog of the BotFramework in a SKFunction, but here we run into the issue. The Semantic Kernel Function expects a result but the "Turn Based" approach of the BotFramework will end the Function without an immediately answer.

You can program around this with stopping the planner, starting a Dialog and then pick the Conversation Up, but since the Stateless Approach of BotFramework and the "keep the whole history" approach in LLM's it is pretty hard to achieve.

What i would like to see

I would like to see a Concept to execute high critical workloads with LLM's with avoiding Hallucinations to 100% here. I think Semantic Kernel and Azure Bot Framework are capable of achieving this together !

Because it could enable the current flow :

  1. User requires critical Workload --> Planner is getting Parameters (Hallucinations possible) --> Bot SDK is starting Dialog to validate Parameters --> User Approves --> Critical Workload is executed

The Final conclusion

I see great potential as synergy between the Products to overcome the Problem that currently hinders to execute critical workloads with LLM's. A concept or example of this would really help the industry to speed up on the Investment in AI Probably !

arafattehsin commented 8 months ago

I came here to see what's happening in this space and glad to see that I am not the only one wanting this! 🙏

huislaw commented 5 months ago

I came here to see what's happening in this space and glad to see that I am not the only one wanting this! 🙏

I want this too.

Hope can do something like this with SK: https://github.com/microsoft/semantic-kernel/discussions/4853

If SK alone cannot, then seamless integration of SK with Microsoft Bot Framework might be the answer.