json_schema support in Dify 0.9.1

[ivanberry]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.9.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

0. Use Azure OpenAI gpt model
1. Create a new workflow
2. use a LLM block
3. Change the model to gpt-4o-mini-2024-07-18 or gpt-4o-2024-08-06 as How to Use JSON Schema Output in Dify?
4. set up the response_format to json_schema and add a json_schema

✔️ Expected Behavior

JSON response

[openai] Bad Request Error, Error code: 400 - {'error': {'message': 'response_format value as json_schema is enabled only for api versions 2024-08-01-preview and later (request id: 2024100312073850903022739187310)', 'type': '', 'param': '', 'code': 'BadRequest'}}

❌ Actual Behavior

Error shows:

How to Use JSON Schema Output in Dify?

[dosubot[bot]]

Hello, @ivanberry! I'm here to help you out with any bugs, questions, or contributions you might have for Dify. Let's work together to solve the issue at hand.

The error message you're encountering indicates that the json_schema response format is only supported for API versions 2024-08-01-preview and later. To resolve this issue, ensure that you are using a compatible API version with your Azure OpenAI model. The supported API versions for using the json_schema response format in Dify 0.9.1 are gpt-4o-mini-2024-07-18 and later versions, and gpt-4o-2024-08-06 and later versions ^[1]. Make sure your setup aligns with these requirements.

To continue talking to Dosu, mention @dosu.

[ivanberry]

As I method in the [Steps to reproduce] section. I am test the gpt-4o-mini-2024-07-18 , and gpt-4o-2024-08-06

[rosehuynh92]

Tự kiểm tra

• [x] Đây chỉ là mục đích báo cáo lỗi, nếu bạn muốn hỏi bất kỳ câu hỏi nào, vui lòng vào mục Thảo luận .
• [x] Tôi đã tìm kiếm các vấn đề hiện có , bao gồm cả những vấn đề đã đóng.
• [x] Tôi xác nhận rằng tôi đang sử dụng tiếng Anh để gửi báo cáo này (我已阅读并同意Chính sách ngôn ngữ ).
• [x] [DÀNH CHO NGƯỜI DÙNG TRUNG QUỐC] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
• [x] Vui lòng không sửa đổi mẫu này :) và hãy điền vào tất cả các trường bắt buộc.

Phiên bản Dify

0.9.1

Đám mây hoặc tự lưu trữ

Tự lưu trữ (Docker)

Các bước để tái tạo

0. Sử dụng mô hình Azure OpenAI gpt
1. Tạo một quy trình làm việc mới
2. sử dụng khối LLM
3. Đổi mô hình thành gpt-4o-mini-2024-07-18 hoặc gpt-4o-2024-08-06 như thế nào để sử dụng đầu ra lược đồ JSON trong Dify?
4. thiết lập response_format thành json_schema và thêm json_schema

✔️ Hành vi mong đợi

Phản hồi JSON

[openai] Lỗi yêu cầu không hợp lệ, Mã lỗi: 400 - {'error': {'message': 'giá trị định dạng phản hồi dưới dạng json_schema chỉ được bật cho các phiên bản api 2024-08-01-preview trở lên (id yêu cầu: 2024100312073850903022739187310)', 'type': '', 'param': '', 'code': 'BadRequest'}}

❌ Hành vi thực tế

Lỗi hiển thị:

Làm thế nào để sử dụng đầu ra lược đồ JSON trong Dify?

[vannhann-intel]

i'm actually having the same issue, previously work just fine. only today have this problem

[crazywoola]

[openai] Bad Request Error, Error code: 400 - {'error': {'message': 'response_format value as json_schema is enabled only for api versions 2024-08-01-preview and later (request id: 2024100312073850903022739187310)', 'type': '', 'param': '', 'code': 'BadRequest'}}

See the docs here.

However, Structured Outputs with response_format: {type: "json_schema", ...} is only supported with the gpt-4o-mini, gpt-4o-mini-2024-07-18, and gpt-4o-2024-08-06 model snapshots and later. gpt-4o-mini, gpt-4o-2024-08-06, and later

https://platform.openai.com/docs/guides/structured-outputs/structured-outputs-vs-json-mode

This error is pretty obvious, you need to change to another model, the docs might be outdated in this case.

For questions please use discussions instead of posting an issue.

[dabrodev]

I switched to another model: gpt-4o-2024-08-06. Yesterday it worked, today it does not.

[ivanberry]

@dabrodev @vannhann-intel

is it worked now?

The comment they response is useless. I know the model limitation. and I have said I am use the model gpt-4o-mini-2024-07-18 or gpt-4o-2024-08-06 as How to Use JSON Schema Output in Dify?.

[ofekby]

@dabrodev @vannhann-intel

is it worked now?

The comment they response is useless. I know the model limitation. and I have said I am use the model gpt-4o-mini-2024-07-18 or gpt-4o-2024-08-06 as How to Use JSON Schema Output in Dify?.

I have experienced a similar issue, not using Dify, but using Azure Open AI. Changing api_version to 2024-08-01-preview, solved the issue. Based on this doc https://devblogs.microsoft.com/semantic-kernel/using-json-schema-for-structured-output-in-net-for-openai-models/#supported-models-for-structured-outputs

Note, you should have both requirements - a model supporting structured responses and use the API version 2024-08-01-preview

[tuoxiansp]

It seems to be a bug in the Azure model provider.

https://github.com/langgenius/dify/issues/8451#issuecomment-2402979446