search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.24k stars 4.58k forks source link

Setting Response Format to JSON for OpenAI Assistant #43315

Open raysuelzer opened 5 months ago

raysuelzer commented 5 months ago

Library name and version

Azure.AI.OpenAI.Assistants_1.0.0-beta.3

Query/Question

I cannot seem to find anywhere in the Assistants SDK to set the "response format" when using the OpenAI SDKs. This is useful to ensure that the API returns valid JSON. Sorry if this exists but I am missing it.

From OpenAI:

https://platform.openai.com/docs/api-reference/chat/create#chat-create-response_format

response_format
object

Optional
An object specifying the format that the model must output. Compatible with [GPT-4 Turbo](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo) and all GPT-3.5 Turbo models newer than gpt-3.5-turbo-1106.

Setting to { "type": "json_object" } enables JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if finish_reason="length", which indicates the generation exceeded max_tokens or the conversation exceeded the max context length.

Environment

No response

github-actions[bot] commented 5 months ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @trrwilson.

raysuelzer commented 3 months ago

Any help on this would be appreciated. I'm having some real issues gettng the assistant to return JSON responses.

nighby commented 3 months ago

@raysuelzer as a temporary workaround, you can visit https://platform.openai.com/assistants using your account (the one tied to the API key you are using to create assistants), click on your assistant and toggle 'JSON object' under Response format

medoni commented 2 months ago

Still missing in 1.0.0-beta.4

TomasHubelbauer commented 2 weeks ago

Ideally, this would also support the json_schema value introduced with structured outputs, not just json_object: https://openai.com/index/introducing-structured-outputs-in-the-api/.

FWIW as of 2.0.0-beta2, neither value is supported.