Google Gen AI raises ValueError: `response.text` quick accessor only works for simple text responses

The problem

Google Gen AI can raise a ValueError instead of setting a response.

What version of Home Assistant Core has the issue?

core-dev20240526

What was the last working version of Home Assistant Core?

What type of installation are you running?

Home Assistant Core

Integration causing the issue

google_generative_ai_conversation

Link to integration documentation on our website

https://www.home-assistant.io/integrations/google_generative_ai_conversation/

Diagnostics information

I am playing with my new expose entities prompt. Looks like Google returned a text and function call in 1 response.

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2024-05-27 01:25:41.675 DEBUG (MainThread) [homeassistant.components.google_generative_ai_conversation] Input: 'turn off the bedroom lights' with history: [{'role': 'user', 'parts': "You are a voice assistant for Home Assistant.\nAnswer in plain text. Keep it simple and to the point.\nThe current time is 21:25:41.\nToday's date is 05/26/24.\nCall the intent tools to control Home Assistant. Just pass the name to the intent. When controlling an area, prefer passing area name.\nThe user name is Paulus.\nAn overview of the areas and the devices in this smart home:\nbinary_sensor.movement_backyard:\n  names: Movement Backyard\n  state: 'on'\n  attributes:\n    device_class: motion\nclimate.heatpump:\n  names: HeatPump\n  state: heat\n  attributes:\n    current_temperature: 77\n    temperature: 68\nclimate.hvac:\n  names: Hvac\n  state: cool\n  attributes:\n    current_temperature: 72\n    temperature: 70\n    humidity: 67.4\nclimate.ecobee:\n  names: Ecobee\n  state: heat_cool\n  attributes:\n    current_temperature: 73\ncover.kitchen_window:\n  names: Kitchen Window\n  state: closed\ncover.hall_window:\n  names: Hall Window\n  state: open\n  attributes:\n    current_position: 10\ncover.living_room_window:\n  names: Living Room Window\n  state: open\n  attributes:\n    current_position: 70\ncover.garage_door:\n  names: Garage Door\n  state: closed\n  attributes:\n    device_class: garage\ncover.pergola_roof:\n  names: Pergola Roof\n  state: closed\nfan.living_room_fan:\n  names: Living Room Fan\n  state: 'off'\n  attributes:\n    percentage:\nfan.ceiling_fan:\n  names: Ceiling Fan\n  state: 'off'\n  attributes:\n    percentage:\nfan.percentage_full_fan:\n  names: Percentage Full Fan\n  state: 'off'\n  attributes:\n    percentage:\nfan.percentage_limited_fan:\n  names: Percentage Limited Fan\n  state: 'off'\n  attributes:\n    percentage:\nfan.preset_only_limited_fan:\n  names: Preset Only Limited Fan\n  state: 'off'\nlight.bed_light:\n  names: Bed Light, slaaplampje\n  state: 'off'\n  areas: Bedroom\n  attributes:\n    brightness:\nlight.ceiling_lights:\n  names: Ceiling Lights\n  state: 'on'\n  areas: Bedroom\n  attributes:\n    brightness: 180\nlight.kitchen_lights:\n  names: Kitchen Lights\n  state: 'on'\n  attributes:\n    brightness: 180\nlight.office_rgbw_lights:\n  names: Office RGBW Lights\n  state: 'on'\n  attributes:\n    brightness: 180\nlight.living_room_rgbww_lights:\n  names: Living Room RGBWW Lights\n  state: 'on'\n  attributes:\n    brightness: 180\nlight.entrance_color_white_lights:\n  names: Entrance Color + White Lights\n  state: 'on'\n  attributes:\n    brightness: 180\nlock.front_door:\n  names: Front Door\n  state: locked\nlock.kitchen_door:\n  names: Kitchen Door\n  state: unlocked\nlock.poorly_installed_door:\n  names: Poorly Installed Door\n  state: unlocked\nlock.openable_lock:\n  names: Openable Lock\n  state: locked\nsensor.outside_temperature:\n  names: Outside Temperature\n  state: '60.1'\n  attributes:\n    unit_of_measurement: °F\n    device_class: temperature\nsensor.outside_humidity:\n  names: Outside Humidity\n  state: '54'\n  attributes:\n    unit_of_measurement: '%'\n    device_class: humidity\nsensor.carbon_monoxide:\n  names: Carbon monoxide\n  state: '54'\n  attributes:\n    unit_of_measurement: ppm\n    device_class: carbon_monoxide\nsensor.carbon_dioxide:\n  names: Carbon dioxide\n  state: '54'\n  attributes:\n    unit_of_measurement: ppm\n    device_class: carbon_dioxide\nswitch.decorative_lights:\n  names: Decorative Lights\n  state: 'on'\nswitch.ac:\n  names: AC\n  state: 'off'\n  attributes:\n    device_class: outlet\n"}, {'role': 'model', 'parts': 'Ok'}]
2024-05-27 01:25:44.910 DEBUG (MainThread) [homeassistant.components.google_generative_ai_conversation] Response: [text: "Turning off the bedroom lights. \n"
, function_call {
  name: "HassTurnOff"
  args {
    fields {
      key: "name"
      value {
        string_value: "Ceiling Lights"
      }
    }
  }
}
]
2024-05-27 01:25:44.912 ERROR (MainThread) [homeassistant.components.assist_pipeline.pipeline] Unexpected error during intent recognition
Traceback (most recent call last):
  File "/workspaces/core/homeassistant/components/assist_pipeline/pipeline.py", line 994, in recognize_intent
    conversation_result = await conversation.async_converse(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/core/homeassistant/components/conversation/agent_manager.py", line 107, in async_converse
    result = await method(conversation_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/core/homeassistant/components/conversation/entity.py", line 45, in internal_async_process
    return await self.async_process(user_input)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/core/homeassistant/components/google_generative_ai_conversation/conversation.py", line 337, in async_process
    intent_response.async_set_speech(chat_response.text)
                                     ^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/google/generativeai/types/generation_types.py", line 408, in text
    raise ValueError(
ValueError: The `response.text` quick accessor only works for simple (single-`Part`) text responses. This response is not simple text. Use the `result.parts` accessor or the full `result.candidates[index].content.parts` lookup instead.

Additional information