jekalmin / extended_openai_conversation

Home Assistant custom component of conversation agent. It uses OpenAI to control your devices.
834 stars 108 forks source link

Google Search not working? #180

Closed alexkahler closed 3 months ago

alexkahler commented 3 months ago

I tried setting up the Google Search function call, but it's returning an error. Something about the JSON package from Google Search not parsing well, if I'm guessing right? I would love some more feedback on this.

2024-03-25` 02:04:42.870 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [140409109869248] Error handling message: Unknown error (unknown_error) REDACTED from 192.168.50.250 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0)
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/homeassistant/components/websocket_api/decorators.py", line 26, in _handle_async_response
await func(hass, connection, msg)
File "/usr/local/lib/python3.12/site-packages/homeassistant/components/websocket_api/commands.py", line 795, in handle_execute_script
script_result = await script_obj.async_run(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 1600, in async_run
return await asyncio.shield(run.async_run())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 435, in async_run
await self._async_step(log_exceptions=False)
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 487, in _async_step
self._handle_exception(
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 512, in _handle_exception
raise exception
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 485, in _async_step
await getattr(self, handler)()
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 723, in _async_call_service_step
response_data = await self._async_run_long_action(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 685, in _async_run_long_action
return long_task.result()
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/core.py", line 2319, in async_call
response_data = await coro
^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/core.py", line 2356, in _execute_service
return await target(service_call)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/components/conversation/__init__.py", line 178, in handle_process
result = await async_converse(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/components/conversation/__init__.py", line 543, in async_converse
result = await agent.async_process(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/__init__.py", line 196, in async_process
query_response = await self.query(user_input, messages, exposed_entities, 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/__init__.py", line 380, in query
return await self.execute_function_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/__init__.py", line 406, in execute_function_call
return await self.execute_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/__init__.py", line 432, in execute_function
result = await function_executor.execute(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/helpers.py", line 496, in execute
value = value_template.async_render_with_possible_json_value(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/template.py", line 753, in async_render_with_possible_json_value
render_result = _render_with_context(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/template.py", line 2364, in _render_with_context
return template.render(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/usr/local/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "<template>", line 1, in top-level template code
TypeError: 'builtin_function_or_method' object is not iterable

I'm using the standard Google Search example (with my own CX and API KEY).

- spec:
    name: search_google
    description: Search Google using the Custom Search API.
    parameters:
      type: object
      properties:
        query:
          type: string
          description: The search query.
      required:
      - query
  function:
    type: rest
    resource_template: "https://www.googleapis.com/customsearch/v1?key=[GOOGLE_API_KEY]&cx=[GOOGLE_PROGRAMMING_SEARCH_ENGINE]:omuauf_lfve&q={{ query }}&num=3"
    value_template: >-
      {% if value_json.items %}
      ```csv
      title,link
      {% for item in value_json.items %}
      "{{ item.title | replace(',', ' ') }}","{{ item.link }}"
      {% endfor %}
  {% else %}
  No results found,
  {% endif %}
Al3XKOoL commented 3 months ago

I want this feature.

jekalmin commented 3 months ago

Can you try this?

- spec:
    name: search_google
    description: Search Google using the Custom Search API.
    parameters:
      type: object
      properties:
        query:
          type: string
          description: The search query.
      required:
      - query
  function:
    type: rest
    resource_template: "https://www.googleapis.com/customsearch/v1?key=[GOOGLE_API_KEY]&cx=[GOOGLE_PROGRAMMING_SEARCH_ENGINE]:omuauf_lfve&q={{ query }}&num=3"
    value_template: >-
      {% if value_json["items"] %}
      ```csv
      title,link
      {% for item in value_json["items"] %}
      "{{ item.title | replace(',', ' ') }}","{{ item.link }}"
      {% endfor %}
  {% else %}
  No results found,
  {% endif %}

Or this?

```yaml
- spec:
    name: search_google
    description: Search Google using the Custom Search API.
    parameters:
      type: object
      properties:
        query:
          type: string
          description: The search query.
      required:
      - query
  function:
    type: rest
    resource_template: "https://www.googleapis.com/customsearch/v1?key=[GOOGLE_API_KEY]&cx=[GOOGLE_PROGRAMMING_SEARCH_ENGINE]&q={{ query }}&num=3"
    value_template: >-
      {% if value_json["items"] %}
      ```csv
      title,link
      {% for item in value_json["items"] %}
      "{{ item.title | replace(',', ' ') }}","{{ item.link }}"
      {% endfor %}
  {% else %}
  No results found,
  {% endif %}
alexkahler commented 3 months ago

The second function worked 👍

OLD COMMENT: Tried calling the first function from Developer Tools > Services

service: conversation.process
data:
  agent_id: 075e43b1a0ff379c69f9ef17385afd9b
  text: Hi chat, what are the latest news headlines?
response_variable: chatgpt

Received error message: Failed to call service conversation.process. Unknown error

System logs:

2024-03-26 16:45:05.735 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [140409278469568] Error handling message: Unknown error (unknown_error) REDACTED from 192.168.50.250 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0)
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/homeassistant/components/websocket_api/decorators.py", line 26, in _handle_async_response
await func(hass, connection, msg)
File "/usr/local/lib/python3.12/site-packages/homeassistant/components/websocket_api/commands.py", line 795, in handle_execute_script
script_result = await script_obj.async_run(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 1600, in async_run
return await asyncio.shield(run.async_run())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 435, in async_run
await self._async_step(log_exceptions=False)
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 487, in _async_step
self._handle_exception(
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 512, in _handle_exception
raise exception
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 485, in _async_step
await getattr(self, handler)()
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 723, in _async_call_service_step
response_data = await self._async_run_long_action(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/script.py", line 685, in _async_run_long_action
return long_task.result()
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/core.py", line 2319, in async_call
response_data = await coro
^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/core.py", line 2356, in _execute_service
return await target(service_call)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/components/conversation/__init__.py", line 178, in handle_process
result = await async_converse(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/components/conversation/__init__.py", line 543, in async_converse
result = await agent.async_process(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/__init__.py", line 196, in async_process
query_response = await self.query(user_input, messages, exposed_entities, 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/__init__.py", line 380, in query
return await self.execute_function_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/__init__.py", line 406, in execute_function_call
return await self.execute_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/__init__.py", line 432, in execute_function
result = await function_executor.execute(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/extended_openai_conversation/helpers.py", line 496, in execute
value = value_template.async_render_with_possible_json_value(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/template.py", line 753, in async_render_with_possible_json_value
render_result = _render_with_context(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/template.py", line 2364, in _render_with_context
return template.render(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/usr/local/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "<template>", line 1, in top-level template code
TypeError: 'builtin_function_or_method' object is not iterable