GenerativeModel does not support multiple grounding tools

[bharath4moltin]

Description of the bug:

We have a datastore that contains the knowledge articles which I would like to ground the model on. In addition, for the things it cannot find on there, I am trying to provide an additional tool using Google Search. However, I get a HTTP400 when I add both the tools. If I take out one of the tools, it returns fine.

google_search_tool = Tool.from_google_search_retrieval(grounding.GoogleSearchRetrieval())
data_store_path = f"projects/{__project_id__}/locations/{__data_store_region__}/collections/default_collection/dataStores/{__maestro_datastore_id__}"
        datastore_tool = Tool.from_retrieval(
            grounding.Retrieval(grounding.VertexAISearch(datastore=data_store_path))
        )
self.tools = [
            datastore_tool, google_search_tool
        ]
self.llm = GenerativeModel(model_name)
user_prompt_content= Content(
            role="user",
            parts=[
                Part.from_text("Who is John Galt?"),
            ],
        )
response = self.llm.generate_content(
            user_prompt_content,
            tools=self.tools,
            generation_config=self.generation_config,
            safety_settings=self.safety_settings,
        )

Actual vs expected behavior:

  File "/Users/dandab/codespaces/moltin/maestro-chat-service/app/routers/chat.py", line 41, in chat_gen
    res = chat_service.chat_gen(chat_request)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dandab/codespaces/moltin/maestro-chat-service/app/services/maestro_chat.py", line 347, in chat_gen
    chat_response = self.chat_gen_service.chat(question=chat_request.question,chat_history=chat_history,output_type=output_type)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dandab/codespaces/moltin/maestro-chat-service/app/services/gemini_generation_service.py", line 156, in chat
    response = self.llm.generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dandab/.pyenv/versions/3.12.2/envs/search-env/lib/python3.12/site-packages/vertexai/generative_models/_generative_models.py", line 407, in generate_content
    return self._generate_content(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dandab/.pyenv/versions/3.12.2/envs/search-env/lib/python3.12/site-packages/vertexai/generative_models/_generative_models.py", line 496, in _generate_content
    gapic_response = self._prediction_client.generate_content(request=request)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dandab/.pyenv/versions/3.12.2/envs/search-env/lib/python3.12/site-packages/google/cloud/aiplatform_v1beta1/services/prediction_service/client.py", line 2125, in generate_content
    response = rpc(
               ^^^^
  File "/Users/dandab/.pyenv/versions/3.12.2/envs/search-env/lib/python3.12/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dandab/.pyenv/versions/3.12.2/envs/search-env/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Request contains an invalid argument.

I would have expected it to use both tools like it normally does but seems to have an issue with multiple tools.

Any other information you'd like to share?

google-generativeai -> 0.6.0 python 3.12