DALL-E 3 Tool Not Working

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

Other

36.79k stars 4.99k forks source link

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.10

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

After merging PR #5067, the DALL-E 3 tool is not functioning correctly. The PR added a style consistency parameter (seed id), but this change seems to have caused the tool to malfunction.

Update to the latest code version, including changes from PR #5067.
Attempt to use the DALL-E 3 tool to generate images.
The tool throws an error.

OpenAI Instruction

Even though I haven't seen seed parameter in the official OpenAI instructions for Create Image, could you please verify it?

✔️ Expected Behavior

The DALL-E 3 tool should be able to generate images normally, and the newly added style consistency parameter should not affect its basic functionality.

❌ Actual Behavior

The DALL-E 3 tool fails to generate images or encounters errors during generation.

The issue you're encountering with the DALL-E 3 tool seems to be related to the seed parameter. According to the Dify codebase, the seed parameter is indeed used in the DallE3Tool class when calling the OpenAI API. The extra_body dictionary includes the seed parameter, which is then passed to the client.images.generate method ^[1].

However, the error message indicates that the seed parameter is not allowed in the request body, which suggests that the OpenAI API does not support this parameter. This discrepancy might be causing the HTTP 400 error.

To resolve this issue, you can try removing the seed parameter from the extra_body dictionary in the DallE3Tool class and see if the tool functions correctly without it. Here is the relevant code snippet for reference:

# set extra body
seed_id = tool_parameters.get('seed_id', self._generate_random_id(8))
extra_body = {'seed': seed_id}

# call openapi dalle3
response = client.images.generate(
    prompt=prompt,
    model='dall-e-3',
    size=size,
    n=n,
    extra_body=extra_body,
    style=style,
    quality=quality,
    response_format='b64_json'
)

You can modify it to:

# set extra body
extra_body = {}

# call openapi dalle3
response = client.images.generate(
    prompt=prompt,
    model='dall-e-3',
    size=size,
    n=n,
    extra_body=extra_body,
    style=style,
    quality=quality,
    response_format='b64_json'
)

This change should prevent the seed parameter from being included in the request, potentially resolving the error.

To continue talking to Dosu, mention @dosu.

langgenius / dify