langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
50.83k stars 7.31k forks source link

When I need to use tools, llm use two tools simultaneously, and then an error occurs. #8400

Closed YYyp99 closed 1 week ago

YYyp99 commented 1 month ago

Self Checks

Dify version

0.8.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When I use the search tool and the web scraping tool, it uses them together, calling both tools simultaneously. Afterwards, the LLM throws an error. [bedrock] Error: ValidationException: A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again. image image

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

dosubot[bot] commented 1 month ago

The error you're encountering, ValidationException: A conversation must alternate between user and assistant roles, suggests that the conversation flow is not adhering to the required sequence of alternating between user and assistant roles. This issue can arise when multiple tools are invoked simultaneously, causing the conversation to break this sequence.

To address this, you can manage the sequence of tool invocations to ensure they do not execute simultaneously. Dify provides a mechanism to handle this through the use of a variable pool, which stores variables, files, etc., generated during the tool's operation. This allows tools to cooperate with each other and avoid conflicts.

For example, you can save the output of one tool to the variable pool and then use it as input for another tool, ensuring a sequential execution:

# Example: Saving the output of the first tool to the variable pool
result.append(self.create_blob_message(blob=b64decode(image.b64_json), 
                                       meta={ 'mime_type': 'image/png' },
                                       save_as=self.VARIABLE_KEY.IMAGE.value))

# Example: Retrieving the output from the variable pool for the second tool
image_binary = self.get_variable_file(self.VARIABLE_KEY.IMAGE)
if not image_binary:
    return self.create_text_message('Image not found, please request user to generate image firstly.')

This approach ensures that tools are invoked in a sequence, preventing simultaneous execution errors and maintaining the required conversation flow [1].

Additionally, ensure that your conversation logic explicitly alternates between user and assistant roles to comply with the validation requirements.

To continue talking to Dosu, mention @dosu.

crazywoola commented 1 month ago

Claude does not support system role. Try to switch to another one.

YYyp99 commented 1 month ago

Claude does not support system role. Try to switch to another one.

I'm using Claude from Bedrock, and this issue is caused by calling two different tools simultaneously, right? When I used it before, writing it this way didn't cause any problems.

crazywoola commented 1 month ago

We have a PR 2 days ago. https://github.com/langgenius/dify/pull/8317 maybe you can check with @ybalbert001

YYyp99 commented 1 month ago

We have a PR 2 days ago. #8317 maybe you can check with @ybalbert001

I'm not using a Bedrock cross-region inference model. image I just tried it on the cloud version as well, and it also experienced this issue where calling two tools simultaneously results in an error.

ybalbert001 commented 1 month ago

@YYyp99 how to reproduce you problem? share DSL? This issue seems have no relationship with Bedrock Cross Region Inference

YYyp99 commented 1 month ago

@YYyp99 how to reproduce you problem? share DSL? This issue seems have no relationship with Bedrock Cross Region Inference

image GoogleSearchPoint is my custom Google search tool, while the other two are official tools. When I ask a question, both tools are used together, causing an error. image