unexpected error during intent recognition for google search

DANgerous25 commented 7 months ago

Hi, I've added the function for google search and included my API key. Everything else seems to be working except google search. When I try to use search I get the same error message "unexpected error during intent recognition".

Tried: gpt-3.5-turbo-1106, gpt-3.5-turbo-0125, gpt-4 (all same result) 150 tokens, 4096 tokens (same result)

Using: Home Asisstant 11.4 Extended OpenAI Conversation 1.02

I'm new to HA so I don't know what I'm doing, hoping it's something simple. Any help appreciated!

Debug output:

init_options:
  start_stage: intent
  end_stage: intent
  input:
    text: google search for best pizza in nyc
  pipeline: 01hnsn6g3t77pdj3mt5czbfczs
  conversation_id: null
stage: done
run:
  pipeline: 01hnsn6g3t77pdj3mt5czbfczs
  language: en
  runner_data:
    stt_binary_handler_id: null
    timeout: 300
events:
  - type: run-start
    data:
      pipeline: 01hnsn6g3t77pdj3mt5czbfczs
      language: en
      runner_data:
        stt_binary_handler_id: null
        timeout: 300
    timestamp: "2024-02-07T05:34:08.143748+00:00"
  - type: intent-start
    data:
      engine: 22ececa870c91d4174c42348d8ef044d
      language: "*"
      intent_input: google search for best pizza in nyc
      conversation_id: null
      device_id: null
    timestamp: "2024-02-07T05:34:08.143791+00:00"
  - type: error
    data:
      code: intent-failed
      message: Unexpected error during intent recognition
    timestamp: "2024-02-07T05:34:09.398735+00:00"
  - type: run-end
    data: null
    timestamp: "2024-02-07T05:34:09.398804+00:00"
intent:
  engine: 22ececa870c91d4174c42348d8ef044d
  language: "*"
  intent_input: google search for best pizza in nyc
  conversation_id: null
  device_id: null
  done: false
error:
  code: intent-failed
  message: Unexpected error during intent recognition

Log:

2024-02-07 15:34:03.968 INFO (MainThread) [custom_components.extended_openai_conversation] Response {'id': 'chatcmpl-8pWjTgiB73Nms5w5f37KqAfAsVYLf', 'choices': [{'finish_reason': 'function_call', 'index': 0, 'message': {'role': 'assistant', 'function_call': {'arguments': '{"query":"best NYC pizza restaurant"}', 'name': 'search_google'}}}], 'created': 1707291243, 'model': 'gpt-3.5-turbo-0125', 'object': 'chat.completion', 'system_fingerprint': 'fp_69829325d0', 'usage': {'completion_tokens': 17, 'prompt_tokens': 2121, 'total_tokens': 2138}}
2024-02-07 15:34:04.183 ERROR (MainThread) [homeassistant.components.assist_pipeline.pipeline] Unexpected error during intent recognition
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 988, in recognize_intent
    conversation_result = await conversation.async_converse(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/conversation/__init__.py", line 467, in async_converse
    result = await agent.async_process(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/extended_openai_conversation/__init__.py", line 187, in async_process
    response = await self.query(user_input, messages, exposed_entities, 0)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/extended_openai_conversation/__init__.py", line 312, in query
    message = await self.execute_function_call(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/extended_openai_conversation/__init__.py", line 357, in execute_function
    result = await function_executor.execute(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/extended_openai_conversation/helpers.py", line 421, in execute
    value = value_template.async_render_with_possible_json_value(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 746, in async_render_with_possible_json_value
    return _render_with_context(self.template, compiled, **variables).strip()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 2305, in _render_with_context
    return template.render(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
TypeError: 'builtin_function_or_method' object is not iterable

DANgerous25 commented 7 months ago

Well I managed to get it to work, but not as the original author intended... after a lot of trial and error, I found that the formatting section of the function was the culprit. If I take away the formatting then it doesn't fail. The output isn't exactly how I want it, but at least I can move forward for now.

I also had to remove :omuauf_lfve from the query line as well. No idea why it was in there. Hopefully the below will work for others (don't forget you will need to set up the search API / search engine ID and replace the values below without square brackets).

- spec:
    name: search_google
    description: Search Google using the Custom Search API.
    parameters:
      type: object
      properties:
        query:
          type: string
          description: The search query.
      required:
      - query
  function:
    type: rest
    resource_template: "https://www.googleapis.com/customsearch/v1?key=[GOOGLE_API_KEY]&cx=[GOOGLE_PROGRAMMING_SEARCH_ENGINE]&q={{ query }}&num=3"
    value_template: "{{ value_json }}"

Risbo6 commented 6 months ago

This seems to do the trick without having your GPT cost go to the roof


- spec:
    name: search_google
    description: Search Google using the Custom Search API.
    parameters:
      type: object
      properties:
        query:
          type: string
          description: The search query.
      required:
      - query
  function:
    type: rest
    resource_template: "https://www.googleapis.com/customsearch/v1?key=[GOOGLE_API_KEY]&cx=[GOOGLE_PROGRAMMING_SEARCH_ENGINE]&q={{ query }}&num=3"
    value_template: "{{ value_json['items'] | map(attribute='snippet') | list }}"

jekalmin / extended_openai_conversation

unexpected error during intent recognition for google search #138