search

jekalmin / extended_openai_conversation

Home Assistant custom component of conversation agent. It uses OpenAI to control your devices.
829 stars 108 forks source link

Something went wrong. Service call requested response data but did not match any entities. #208

Closed NG-Bullseye closed 1 month ago

NG-Bullseye commented 1 month ago

Issue Description:

I am experiencing an issue with Home Assistant where I cannot retrieve calendar events for the next day using the calendar.list_events service, despite being able to list all entities and verify that the Google API is connected. The error message I receive is: "Something went wrong, service called requested response data but did not match any entities."

Steps to Reproduce:

  1. Ensure the Google API is connected and functional within Home Assistant.
  2. Confirm that calendar entities can be listed using an entity listing function as described below.
  3. Attempt to retrieve events for the next day using the get_events function.

Expected Behavior:

The get_events function should return calendar events for the specified time period without errors.

Actual Behavior:

When attempting to retrieve events for the next day, the system responds with an error indicating that no matching entities were found, even though the entities appear to be listed without issue in other contexts.

Relevant Configuration:

Here is the configuration for the functions involved in this issue:

- spec:
    name: get_attributes
    description: Get attributes of any home assistant entity
    parameters:
      type: object
      properties:
        entity_id:
          type: string
          description: entity_id
      required:
      - entity_id
  function:
    type: template
    value_template: "{{ states[entity_id] }}"

- spec:
    name: get_events
    description: Use this function to get list of calendar events.
    parameters:
      type: object
      properties:
        start_date_time:
          type: string
          description: The start date time in '%Y-%m-%dT%H:%M:%S%z' format
        end_date_time:
          type: string
          description: The end date time in '%Y-%m-%dT%H:%M:%S%z' format
      required:
      - start_date_time
      - end_date_time
  function:
    type: script
    sequence:
    - service: calendar.list_events
      data:
        start_date_time: "{{ start_date_time }}"
        end_date_time: "{{ end_date_time }}"
      target:
        entity_id: calendar.sehrwichtig
      response_variable: _function_result

Additional Information:

jleinenbach commented 1 month ago
  1. Check if the calendar is exposed.
  2. Use /developer-tools/service to test your calendar calendar.sehrwichtig and see if it works
  3. Check the home-assistant.log file if you see more details.
  4. You use an older example. Try this version as calendar.list_events is deprecated: https://github.com/jekalmin/extended_openai_conversation/tree/main/examples/function/calendar

image

NG-Bullseye commented 1 month ago

Thank you for the guidance and suggestions. I have tested both calendar.list_events and calendar.get_events services and they initially seemed to work correctly in the Developer Tools. However, I encountered an error indicating "Template rendered invalid entity IDs." when using the newer calender.get_events function.

Upon examining the response and the list of available entities, it appears there might be a mistake in the entity ID referenced in my templates. The error message specified calendar.[YourCalendarHere], which gpt4-turbo does not seem to replace with the actual calendar entity ID.

The calenders are exposed and can be displayed when I ask for it. I noticed that in the calendar that I am testing with, legendär wichtig Kalender - Calendar (currently off) with the entity name calendar.sehrwichtig it says "currently off" but it is active! I activate the calendar entity in general, andexposed for the assistant. 

Me: Whats up for tomorrow?

Assistent: Something went wrong:  Template rendered invalid entity IDs: ['calendar.[YourCalendarHere]', 'calendar.[MoreCalendarsArePossible]']

Me: tell me all exposed entities

Assistent: Here are all the exposed entities in your smart home system: 1. **Helix** - Light (currently off) 2. **Forecast Home** - Weather (currently cloudy) 3. **.events** - Calendar (currently off) 4. **verabredet Kalender** - Calendar (currently off) 5. **Anna und leo** - Calendar (currently off) 6. **Geburstage Kalender** - Calendar (currently off) 7. **wichtig Kalender** - Calendar (currently off) 8. **Feiertage in deutschland** - Calendar (currently on) 9. **Todoist Kalender** - Calendar (currently off) 10. **legendär wichtig Kalender** - Calendar (currently off) If you need more details about any specific device or entity, feel free to ask!

When I execute the service manually

Service: calender.list_events Target: calender.sehrwichtig Start: 6.5.2024 End: 8.5.2024

events:
  - start: "2024-05-06T12:00:00+02:00"
    end: "2024-05-06T13:00:00+02:00"
    summary: Fishstone
  - start: "2024-05-06T14:00:00+02:00"
    end: "2024-05-06T18:00:00+02:00"
    summary: Haferkater
  - start: "2024-05-07T14:00:00+02:00"
    end: "2024-05-07T18:00:00+02:00"
    summary: Haferkater

Service: calender.get_events Target: calender.sehrwichtig Start: 6.5.2024 End: 8.5.2024

calendar.sehrwichtig:
  events:
    - start: "2024-05-06T12:00:00+02:00"
      end: "2024-05-06T13:00:00+02:00"
      summary: Fishstone
    - start: "2024-05-06T14:00:00+02:00"
      end: "2024-05-06T18:00:00+02:00"
      summary: Haferkater
    - start: "2024-05-07T14:00:00+02:00"
      end: "2024-05-07T18:00:00+02:00"
      summary: Haferkater

How can I access and interpret the debug information from home-assistant.log to better diagnose the issues?

jleinenbach commented 1 month ago

  1. Initiate a backup, then add this to your configuration.yaml, then restart:

    logger:
logs:
custom_components.extended_openai_conversation: info

  2. I noticed that ChatGPT 3.5 fails to set the parameters correct, but only with arrays/lists. So the logs are very useful to check this.

  3. ['calendar.[YourCalendarHere]', 'calendar.[MoreCalendarsArePossible]'] Did you replace these values with your static calendars?

jleinenbach commented 1 month ago

@NG-Bullseye Or you may want to try my version which supports labels. E.g. use a calendar label to check all calendars at once.

- spec:
    name: get_events
    description: Use this function to get a list of calendar events.
    parameters:
      type: object
      properties:
        target:
          type: object
          properties:
            entity_id:
              type: string
              description: The entity_id of the calendar. Either entity_id or label_id is required.
            label_id:
              type: string
              description: The label_id of calendars. Either entity_id or label_id is required.
          anyOf:
            - required: ["entity_id"]
            - required: ["label_id"]
        start_date_time:
          type: string
          description: The start date time in '%Y-%m-%dT%H:%M:%S%z' format.
        final_time:
          type: object
          properties:
            end_date_time:
              type: string
              description: The end date time in '%Y-%m-%dT%H:%M:%S%z' format.
            duration:
              type: object
              description: Return active events from start_date_time until the specified duration. Expressed as a dictionary with days, hours, or minutes.
          oneOf:
            - required: ["end_date_time"]
            - required: ["duration"]
      required:
        - start_date_time
        - final_time
        - target
  function:
    type: composite
    sequence:
      - type: script
        sequence:
          - variables:
              target_data: >
                {% set data = {} %}
                {% if target.entity_id is defined and has_value(target.entity_id) and 'entity_id' in target and target.entity_id.split('.')[0] == 'calendar' %}
                  {% set data = dict(data, **{'entity_id': target.entity_id}) %}
                {% endif %}
                {% if target.label_id|default('') in labels() %}
                  {% set data = dict(data, **{'label_id': target.label_id}) %}
                {% endif %}
                {{ data | tojson }}
              event_data: >
                {% set data = {} %}
                {% set data = dict(data, **{'start_date_time': start_date_time}) %}
                {% if 'end_date_time' in final_time and final_time.end_date_time %}
                  {% set data = dict(data, **{'end_date_time': (final_time.end_date_time | default(none))}) %}
                {% elif 'duration' in final_time and final_time.duration %}
                  {% set data = dict(data, **{'duration': (final_time.duration | default(none))}) %}
                {% endif %}
                {{ data | tojson }}
              data_filled: >
                {% set target_dict = (target_data | from_json) if target_data is string else target_data %}
                {% set event_dict = (event_data | from_json) if event_data is string else event_data %}
                {{ target_dict | length > 0 and event_dict | length > 0 }}
          - condition: template
            value_template: "{{ data_filled }}"
          - service: calendar.get_events
            data: "{{ event_data }}"
            target: "{{ target_data }}"
            response_variable: _function_result
NG-Bullseye commented 1 month ago

Thank you for your response.

At the moment, with the Docker configuration in a container of Home Assistant, I am unable to access the configuration YAML file directly. I will set this up when I have time, so I haven't been able to test the logging information yet.

To your question: I did not replace these values with static calendars. My understanding was that the placeholders would be filled by the entities found via the GetAttributes specification.

RANDOM CALENDER EVENTS: I tried your configuration, but it responded in an unexpected manner. It initially claimed there were no appointments for tomorrow. While the previous error is gone, which is an improvement, the system then provided two appointments not present in my calendar.

Moreover, these calendar names are not on any other day either, and there are no events in my entire calendar named as indicated.

Also, I tried referring to the Home Assistant documentation for guidance on how to interpret these YAML files. I read through templating and other documentations but couldn't find helpful documentation on how to interpret and tinker these function configuration YAML files that configure the calendar calls and other GPT_extended functionalities. Do you have a resource for this?

As a resource, here is the current conversation.

whats up for tomorrow? 

For tomorrow, the following appointments are scheduled in your calendars: 
1. Board Meeting - from 10:30 AM to 12:00 PM. 
2. Project Discussion - from 2:00 PM to 3:30 PM. Both appointments are listed in the "important calendar". Do you need more details about these events or is there anything else I can assist you with?

Thanks for your help!

jleinenbach commented 1 month ago

ChatGPT 3.5 tends to hallucinate if the result is empty. In this case, my function should answer that no entry could be found. But that's too complicated for now.

Just begin with this example: https://github.com/jekalmin/extended_openai_conversation/tree/main/examples/function/calendar

Then manually replace the examples with your calendar entity or entitites:

The documentation of the function syntax of Extended OpenAI conversation consists of the examples and the explanations of the README file. Based on the example functions, you can use Home Assistant templates, Home Assistant scripts and some native functions.

NG-Bullseye commented 1 month ago

Thank you very much, this seems to work now. The issue seemed to be the following:

instead of 

target:
   entity_id: calendar.sehrwichtig

it should have been

target:
   entity_id:
        - calendar.sehrwichtig