search

phidatahq / phidata

Build AI Agents with memory, knowledge, tools and reasoning. Chat with them using a beautiful Agent UI.
https://docs.phidata.com
Mozilla Public License 2.0
15.63k stars 2.15k forks source link

Made changes to support structured output from OpenAI models along with tools calling #1358

Closed gauravdhiman closed 3 weeks ago

gauravdhiman commented 1 month ago

All this is in context of OpenAI models. Because of big test run logs, I am putting them in pastebin and linking them here, else this description becomes too big and github does not allow too big descriptions.

Details:

As of now structured output works, but only in case if agent is not given any tools. If we give tools to an agent and also expect it to give structured output, it fails with below error: https://pastebin.com/Ckgi1YCU

Here is the test script: https://pastebin.com/72SgZ00j

When either tools are not provided or below two arguments are not passed (or commented) in call to Agent(), the test pass without any issues. 

    structured_outputs=True,
    response_model=ResponseObj,

As from error, we can see that OpenAI expects functions to be marked as "strict": True when structured output is enabled. This diff is to make that change. With this diff now tools along with structured output can be used.

Here is the test run for tools with structured output: https://pastebin.com/xFYcfV8M

Here is the test run for tools without structured output (as markdown): https://pastebin.com/rEwcgrBA