search

metaskills / experts

Experts.js is the easiest way to create and deploy OpenAI's Assistants and link them together as Tools to create advanced Multi AI Agent Systems with expanded memory and attention to detail.
https://www.unremarkable.ai/experts/
MIT License
1.01k stars 58 forks source link

Bespoke UI with Structured Output #24

Open metaskills opened 3 months ago

metaskills commented 3 months ago

So besides this work already done:

Can we do something with the new OpenAI Structured Outputs. See their "Dynamically generating user interfaces based on the user’s intent" section.

https://openai.com/index/introducing-structured-outputs-in-the-api/

Here is a gist of what they showed and how to instruct zod with the new response format helper.

https://gist.github.com/metaskills/2bf51e5f4ed89a5d5e0d3dfe78be0d1f

metaskills commented 3 months ago

I think the question is what processes the output and creates styled presentation layer components.

Landing Page Sign Up Stock Price
Screenshot 2024-08-06 at 4 42 57 PM Screenshot 2024-08-06 at 4 43 04 PM Screenshot 2024-08-06 at 4 43 18 PM

Their demos do not seem to line up with the shown zod response format. Which is expected. I get the point. But still there are a lot of questions. For example, are we going to throw a bunch of HTML into the context window from a tool output and expect the LLM to add it to a chat completion or assistants response verbatim?

Screenshot 2024-08-06 at 4 44 10 PM

metaskills commented 3 months ago

Demonstrate Function Calling & Structured Outputs with Zod - https://github.com/metaskills/experts/commit/ef54d182936f1836c512de440a54baf564ffdd43