Closed arajawat closed 8 months ago
@arajawat, is the model being served up by OpenAI or Azure OpenAI? Also.. are you suddently getting a 400 error for other types of prompts that you use with your model? Or is it specific to the planner?
@markwallace-microsoft or @alliscode, are either of y'all aware of a change in the endpoints of OpenAI or Azure OpenAI that would cause this to suddenly break?
@matthewbolanos We're using Azure OpenAI. It is specific to only FunctionCallingStepwisePlanner only. I am able to call HandleBarsPlanner and get the desired result and I am also able to directly call kernel, pass the prompt and get the result.
@matthewbolanos The issue seems to be with Azure OpenAI deployment for models "gpt-4" & "gpt-4-32k". We are not getting any error "gpt-35-turbo-16k" and everything works fine with this.
The above mentioned error is coming from model version - 1106-Preview
For other model version "0613" of gpt-4 and gpt-4-32k, we get some other error stated below:
{ "error": { "message": "Unrecognized request arguments supplied: tool_choice, tools", "type": "invalid_request_error", "param": null, "code": null } }
I can confirm, there is a issue in version 1.0.1 but I just download sourse code, and seems now gpt4-turbo works fine.
There have been some changes to ClientCore.cs
in the past few days, so I'm wondering if there's a connection there. I have not been able to reproduce this issue myself on the latest code.
@arajawat @KSemenenko I would love to get a repro of this and attach a breakpoint to see what the chat history looks like at the time/just before the exception is thrown. If either of you have a set of reliable repro steps that you can share, that would be really helpful!
Edit: I was using gpt-35-turbo-16k, which doesn't repro the issue. I'll give it a try with one of the other models.
I am now able to reproduce the issue. Investigating....
This appears to be an issue with the model on Azure OpenAI service, not in our code.
I can see that the model is returning a response with finish_reason
equal to tool_calls
, but the list of tool calls is empty and the content string is null (as it should be for a tool call). When we add this message back into chat history and request another completion, the model identifies this entry in the chat history as invalid, since neither the tool call or content are populated.
At this time, the only workaround I can suggest is to use a different model version -- this seems to be an issue with the 1106 models.
Here's a related thread on the OpenAI community forum: https://community.openai.com/t/function-call-response-is-empty-despite-completion-tokens-being-used/580888
Sorry, forgot to mentioned, for me it was Azure OpenAI
@gitri-ms Thanks for the analysis and sharing associated thread. Apparently there are 2 different issues with both available versions:
Issue with 1106 preview (& you provided your research insights above) : Invalid value for 'content': expected a string, got null. Issue with 0613 : Unrecognized request arguments supplied: tool_choice, tools
So even the other "0613" model version is throwing error. For now, the only work around is stepping back to gpt35-turbo-16k.
Is this error depending on the Azure region? This error occurs in East US 2 region, but works in Sweden Central region with 0613.
Dupe of #4674 which has been resolved
Describe the bug Using "FunctionCallingStepwisePlanner" is throwing error while calling ExecuteAsync. The error response looks like this -
{ "message": "Invalid value for 'content': expected a string, got null.", "type": "invalid_request_error", "param": "messages.[18].content", "code": null }
The ExecuteAsync call was working normally atleast till 08Jan'24 but stopped working on 9th. I'm still able to create and invoke the plan using HandleBarsPlanner correctly and use kernel.InvokeAsync as well.
To Reproduce Steps to reproduce the behavior:
var planner=new FunctionCallingStepwisePlanner();
var result = await planner.ExecuteAsync(kernel, planPrompt);
Expected behavior Expected a properly formed response JSON.
Screenshots
Platform
nuget - Microsoft.SemanticKernel.Planners.OpenAI version: 1.0.1-preview Microsoft.SemanticKernel version: 1.0.1 Model name: gpt-4 Model version: 1106-Preview
Additional context The issue suddenly started happening while planner was working pretty fine for last few weeks.