jekalmin / extended_openai_conversation

Home Assistant custom component of conversation agent. It uses OpenAI to control your devices.
839 stars 112 forks source link

Incorrect example in the documentation #126

Open RadzuPL opened 5 months ago

RadzuPL commented 5 months ago

In the documentation at: https://github.com/jekalmin/extended_openai_conversation/blob/main/examples/function/weather/README.md#function The get_attributes function is pasted instead of weather function.

jekalmin commented 5 months ago

Thanks for reporting an issue!

This was intended that asking weather can be achieved using get_attributes function. However, as you said, it seems like typo. I will fix README.

TBTHEDBA commented 5 months ago

What weather services do you use? I am getting false returns from both weather.openweathermap and weather.pirateweather

jekalmin commented 5 months ago

Although I'm using naver_weather which is designed to support Korea only, weather entity should have the same format, I guess, like below.

스크린샷 2024-02-01 오전 11 32 55

Have you exposed weather.xxx entity?

TBTHEDBA commented 5 months ago

It is exposed and says that it is referencing the correct Entity when I ask where it derived it's data from. But it keeps giving 25F (-4C) as the forecasted temperature for any request (Example: What will the temp be at 7am tomorrow / What will the coldest day be next week?) when the temps are more in the 60F (15C) range.

TBTHEDBA commented 5 months ago

Update: I think I found the issue. I had both the original function for weather and get attribute. Once I removed the original function, I am getting correct results.

TBTHEDBA commented 5 months ago

Further update: That wasn't the issue the symptom has returned.


Assist: How can I assist?

Me: What will the temp be at 8am tomorrow?

Assist:I'm unable to provide the weather forecast for tomorrow at 8 am without access to an external weather service. If you have a weather sensor integrated into your smart home system, I can retrieve the current temperature for you.

Me: why didn't you use weather.openweathermap?

Assist: My apologies for the oversight. The forecasted temperature for tomorrow at 8 am is approximately 62.73°F based on the OpenWeatherMap integration in your smart home system.


It is giving me the current weather. Sometimes it will look at the forecast and other times return current temp as forecasted temp.

TBTHEDBA commented 5 months ago

I can get proper returns from my chatgpt 4 customGPT

image

jekalmin commented 5 months ago

When asked about forecast, it seems gpt doesn't call get_attribute function of weather entity. In my case, it calls get_events function instead.

I think this is because gpt calls date or time relevent functions if exist rather than calls get_attribute of weather entity. If I ask How is the weather? and then ask about forecast seems working for me.

Scags104 commented 4 months ago

use this as a function. Seems to work for me where there are updated weather services to the new HA spec where service call to get_forecasts is needed:

- spec:
    name: get_weather_forecast
    description: get weather forecast
    parameters:
      type: object
      properties:
        type:
          type: string
          description: the type of forecast
          enum:
            - daily
            - hourly
      required:
      - type
  function:
    type: script
    sequence:
    - service: weather.get_forecasts
      data:
        type: "{{type}}"
      target:
        entity_id: weather.your_weather_entity
      response_variable: _function_result

image

For reference, im using Pirate Weather as my weather service.

TBTHEDBA commented 4 months ago

Could you repost that with the code function to keep formatting?

TBTHEDBA commented 4 months ago

I think if you have it copied with the formatting (say from your notepad++) then it keeps that formatting if you select code block.

Scags104 commented 4 months ago

that didnt work but i did figure it out. 3 ticks on each end

test code

TBTHEDBA commented 4 months ago

It's working great! image image

Scags104 commented 4 months ago

good to hear!

can we add this to the weather example? being lazy and dont want to PR so you can take all the credit :)

Scags104 commented 4 months ago

@TBTHEDBA can you test something? Ask it "what is the weather in 3 hours"

when i do it i get a WILD token error: Sorry, I had a problem talking to OpenAI: Error code: 400 - {'error': {'message': "This model's maximum context length is 16385 tokens. However, your messages resulted in 23954 tokens (23269 in the messages, 685 in the functions). Please reduce the length of the messages or functions.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}

TBTHEDBA commented 4 months ago

Same response as you.

On Thu, Feb 15, 2024 at 2:55 PM Scags @.***> wrote:

@TBTHEDBA https://github.com/TBTHEDBA can you test something? Ask it "what is the weather in 3 hours"

when i do it i get a WILD token error: Sorry, I had a problem talking to OpenAI: Error code: 400 - {'error': {'message': "This model's maximum context length is 16385 tokens. However, your messages resulted in 23954 tokens (23269 in the messages, 685 in the functions). Please reduce the length of the messages or functions.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}

— Reply to this email directly, view it on GitHub https://github.com/jekalmin/extended_openai_conversation/issues/126#issuecomment-1947324680, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJXIGPWPJTDTSHJK5YTKQCLYTZY2LAVCNFSM6AAAAABCRH2AMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBXGMZDINRYGA . You are receiving this because you were mentioned.Message ID: @.***>

Scags104 commented 4 months ago

@jekalmin can you comment on what the issue may be given the function posted above and the token error?

Thanks

TBTHEDBA commented 4 months ago

I switched the model to gpt-4-0125-preview and got the proper response back.

Scags104 commented 4 months ago

tried that model but get a different error now:

Sorry, I had a problem talking to OpenAI: Error code: 429 - {'error': {'message': 'Request too large for gpt-4-turbo-preview in organization org-SAwImXKORCENZ8CcLdMW3snF on tokens per min (TPM): Limit 10000, Requested 15657. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}

jekalmin commented 4 months ago

Thanks @Scags104 for your help!

I wasn't aware that weather.get_forecasts service exists. I just installed and tried pirate weather, and it's returning hourly data from now to a week later, which is huge (about 19,000 tokens)

service call tokens
스크린샷 2024-02-16 오후 11 39 51 스크린샷 2024-02-16 오후 11 34 36

Since we need only one row of data, we have to narrow all hourly data into one to reduce token size.

- spec:
    name: get_weather_forecast
    description: get weather forecast
    parameters:
      type: object
      properties:
        datetime:
          type: string
          description: The date time like "2024-02-16T14:00:00+00:00".
        type:
          type: string
          description: the type of forecast
          enum:
            - daily
            - hourly
      required:
      - type
  function:
    type: composite
    sequence:
      - type: script
        sequence:
          - service: weather.get_forecasts
            data:
              type: "{{type}}"
            target:
              entity_id: weather.pirateweather
            response_variable: _function_result
        response_variable: result
      - type: template
        value_template: "{{result['weather.pirateweather'].forecast | selectattr('datetime', 'contains', datetime) | list}}"

It should be something like above.

  1. Get datetime that we are interested.
  2. Compare it with datetime of forecasts to find weather at specific time.

Although it works with hourly forecast, it doesn't work with daily forecast in my case. The daily forecast is returning 15:00:00 rather than 00:00:00. (I'm located in +9 hours zone) I'm still concerning how to resolve this issue.

1
Scags104 commented 4 months ago

tried yours with no success either, then i tried limiting yours to hours and mine to days to cover both scenarios, no luck there either. A mix of Tool Errors, script errors, bad yaml errors (of my own making) and not picking up either function at all and telling me it cant predict the weather.

Youre definately onto something with the heafty amount of data though. no wonder its not working.

jekalmin commented 4 months ago

This is what I currently have. Please try this out.

- spec:
    name: get_weather_forecast
    description: get weather forecast
    parameters:
      type: object
      properties:
        datetime:
          type: string
          description: The date time like "2024-02-16T14:00:00+00:00".
        type:
          type: string
          description: the type of forecast
          enum:
            - daily
            - hourly
      required:
      - type
  function:
    type: composite
    sequence:
      - type: script
        sequence:
          - service: weather.get_forecasts
            data:
              type: "{{type}}"
            target:
              entity_id: weather.pirateweather
            response_variable: _function_result
        response_variable: result
      - type: template
        value_template: >-
          {%- set forecasts = result['weather.pirateweather'].forecast | selectattr('datetime', 'contains', as_timestamp(datetime) | timestamp_custom('%Y-%m-%dT%H:00:00+00:00', false, 'UTC')) | list %}
          {%- for forecast in forecasts %}
          {{ dict(forecast, datetime=forecast.datetime | as_timestamp | timestamp_custom('%Y-%m-%dT%H:%M:%S')) }}
          {%- endfor %}
Scags104 commented 4 months ago

hourly works great. similarly to your screenshot above, but daily is not working. can not see future daily forecasts

do we need 2 separate functions?

jekalmin commented 4 months ago

No, just one function.

Scags104 commented 4 months ago

go figure, now its working! Maybe i mistyped something before

Thank you!

scottdotau commented 1 month ago

Hey guys,

Was having some issues with this (would throw an error) when asking something like 'whens it going to rain', please see extra function below which has been modified based on above!

Screenshot_20240529-211229_Home Assistant.png

This can probably be turned into a composite (daily/hourly/event) but this works for me (requires integration that works with get_forecasts - change below):

- spec:
    name: get_weather_forecast
    description: get weather forecast
    parameters:
      type: object
      properties:
        datetime:
          type: string
          description: The date time like "2024-02-16T14:00:00+00:00".
        type:
          type: string
          description: the type of forecast
          enum:
            - daily
            - hourly
      required:
      - type
  function:
    type: composite
    sequence:
      - type: script
        sequence:
          - service: weather.get_forecasts
            data:
              type: "{{type}}"
            target:
              entity_id: weather.pirateweather
            response_variable: _function_result
        response_variable: result
      - type: template
        value_template: >-
          {%- set forecasts = result['weather.pirateweather'].forecast | selectattr('datetime', 'contains', as_timestamp(datetime) | timestamp_custom('%Y-%m-%dT%H:00:00+00:00', false, 'UTC')) | list %}
          {%- for forecast in forecasts %}
          {{ dict(forecast, datetime=forecast.datetime | as_timestamp | timestamp_custom('%Y-%m-%dT%H:%M:%S')) }}
          {%- endfor %}

- spec:
    name: get_weather_events
    description: get forecasted weather events such as rain, thunder, sun, clouds, etc
    parameters:
      type: object
      properties:
        event:
          type: string
          description: The type of weather event such as rain, thunder, sun, clouds, etc. 
      required:
      - event
  function:
    type: composite
    sequence:
      - type: script
        sequence:
          - service: weather.get_forecasts
            data:
              type: "hourly"
            target:
              entity_id: weather.pirateweather
            response_variable: _function_result
        response_variable: result
      - type: template
        value_template: >-
          {%- set forecasts = result['weather.pirateweather'].forecast | selectattr('condition', 'contains', event) | list %}
          {%- for forecast in forecasts %}
          {{ dict(forecast, datetime=forecast.datetime | as_timestamp | timestamp_custom('%Y-%m-%dT%H:%M:%S')) }}
          {%- endfor %}