search

microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
22.01k stars 3.28k forks source link

Tool call information missing in Model Diagnostics for Mistral streaming #6316

Closed TaoChenOSU closed 2 months ago

TaoChenOSU commented 6 months ago

Describe the bug Tool call information in model diagnostics when using Mistral streaming.

Expected behavior Tool call information should be presented in model diagnostics when using Mistral streaming.

markwallace-microsoft commented 5 months ago

@TaoChenOSU can we close this issue?

TaoChenOSU commented 2 months ago

Will do another round of testing.

TaoChenOSU commented 2 months ago

Tool calls from MistralAI will be contained entirely in a single chunk returned from their streaming API, thus it doesn't require buffering during streaming and concatenation after streaming to reassemble the complete tool call. That being said, the current implementation of the MistralAI streaming chat completion service in .Net SK doesn't need any modification to have tool calls appear in the prompt span events. Image Note: we are not sure if MistralAI supports parallel function calling and if it does how will multiple tool calls get streamed back to the client.