Google Gemini provider causes error when the same tool is called twice at the same time

[chazzhou]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.10

Cloud or Self Hosted

Cloud, Self Hosted (Docker), Self Hosted (Source)

Steps to reproduce

1. Use the Google Gemini 1.5 model provider in Agent mode.
2. Observe that the model can output two identical tool calls at once.

You can easily replicate this with the following DSL with Gemini 1.5 Pro:

app:
  description: ''
  icon: scream
  icon_background: '#FFEAD5'
  mode: agent-chat
  name: POC
model_config:
  agent_mode:
    enabled: true
    max_iteration: 5
    strategy: function_call
    tools:
    - enabled: true
      isDeleted: false
      notAuthor: false
      provider_id: jina
      provider_name: jina
      provider_type: builtin
      tool_label: JinaReader
      tool_name: jina_reader
      tool_parameters:
        summary: ''
        target_selector: ''
        url: ''
        wait_for_selector: ''
  annotation_reply:
    enabled: false
  chat_prompt_config: {}
  completion_prompt_config: {}
  dataset_configs:
    datasets:
      datasets: []
    retrieval_model: single
  dataset_query_variable: ''
  external_data_tools: []
  file_upload:
    image:
      detail: high
      enabled: false
      number_limits: 3
      transfer_methods:
      - remote_url
      - local_file
  model:
    completion_params:
      stop: []
    mode: chat
    name: gemini-1.5-pro-latest
    provider: google
  more_like_this:
    enabled: false
  opening_statement: What would you like to do?
  pre_prompt: 'Please follow the tasks below:

    ### Tasks:

    1. Use the jina_reader tool to gather relevant information about the latest news
    from the url: "https://www.cnn.com/".

    2. Use the jina_reader tool to gather relevant information about the latest news
    from the url: "https://www.bbc.com/".

    3. After the two tool calls are returned, answer the user''s query to the best
    of your abilities. '
  prompt_type: simple
  retriever_resource:
    enabled: true
  sensitive_word_avoidance:
    configs: []
    enabled: false
    type: ''
  speech_to_text:
    enabled: false
  suggested_questions:
  - Please perform the tasks.
  suggested_questions_after_answer:
    enabled: true
  text_to_speech:
    enabled: false
    language: ''
    voice: ''
  user_input_form: []

✔️ Expected Behavior

• The tool calling result should perform normally, as with the OpenAI provider (see screenshots).

❌ Actual Behavior

• The api\core\agent\fc_agent_runner.py fails to process the function calls and save the agent thought correctly.
• The front end only shows the later call and encounters an error (fixed by PR #5068).
• The function calling result is passed to the LLM correctly, but the saved thought is incorrect.

Additional info:

• The issue lies in the FunctionCallAgentRunner(BaseAgentRunner) class, specifically the run() function.
• Fixing this issue requires major refactoring, cleanup, and reorganization of the run() function's logic.

Suggestion: I'm happy to submit a PR to address this issue, but I believe it would be best handled by the Dify team members.

Please let me know if you need any further information or assistance.

[dosubot[bot]]

To resolve the issue with the Google Gemini provider in Dify version 0.6.10, where calling the same tool twice at the same time causes an error, you need to ensure that the tool runtime is properly managed to handle concurrent invocations. Implementing a locking mechanism or using asynchronous calls can help manage concurrent tool invocations.

Here is a possible approach using a threading lock to ensure that the tool is not invoked concurrently:

from typing import Any
from threading import Lock

from core.tools.errors import ToolProviderCredentialValidationError
from core.tools.provider.builtin.google.tools.google_search import GoogleSearchTool
from core.tools.provider.builtin_tool_provider import BuiltinToolProviderController

class GoogleProvider(BuiltinToolProviderController):
    _lock = Lock()

    def _validate_credentials(self, credentials: dict[str, Any]) -> None:
        with self._lock:
            try:
                GoogleSearchTool().fork_tool_runtime(
                    runtime={
                        "credentials": credentials,
                    }
                ).invoke(
                    user_id='',
                    tool_parameters={
                        "query": "test",
                        "result_type": "link"
                    },
                )
            except Exception as e:
                raise ToolProviderCredentialValidationError(str(e))

This code introduces a threading lock (_lock) to ensure that the GoogleSearchTool is not invoked concurrently. The with self._lock statement ensures that the block of code is executed by only one thread at a time, thus preventing concurrent invocations ^[1].

To continue talking to Dosu, mention @dosu.