Open SergeyMenshykh opened 3 days ago
Here are some early testing results I'd like to share with anybody who wants to look at this.
Switching the strict mode to true pushes additional requirements on the schem:
This last one is really problematic in a scenario where you have more parameters than there's context for. E.g. with GET https://graph.microsoft.com/v1.0/me/messages with supports:
And the prompt using only the subject, recap my last 5 emails ordered by created date time
, the model only has enough information for top/select/orderby, and might be able to figure out to set skip to 0. Gpt4 then gives up saying it couldn't call the function, and doesn't do a better job with the orderby parameter formatting, gpt4o defaults all others to the default for the type, which ends up failing on the service since the request doesn't make sense.
Context: {Azure}OpenAI support the 'strict' mode for function calling that tells the model to adhere to JSON schema of the function definition: Source: https://platform.openai.com/docs/guides/function-calling
Enabling this mode should ensure that the arguments generated by the model for a function call exactly match the JSON Schema provided in the function definition.
ToDo: Add the new
AllowStrictSchemaAdherence
property to theFunctionChoiceBehaviorOptions
class allowing activation of the strict mode for function calling.