home-assistant / intents

Intents to be used with Home Assistant
https://developers.home-assistant.io/docs/voice/overview/
Creative Commons Attribution 4.0 International
463 stars 531 forks source link

Assist not picking up get temperature in area intent #1351

Open blacknell opened 1 year ago

blacknell commented 1 year ago

I've asked what is the temperature in the bathroom and it comes back with Temperature is

But asking what is the bathroom temperature works with Bathroom temperature is 17.9 °C

The problematic question pipeline seems to pick up the domain and area correctly but not the entity. I've looked through the sentences and can't be sure which it is using HassGetState or HassClimateGetTemperature

Problematic language processing

intent_output:
  response:
    speech:
      plain:
        speech: Temperature is
        extra_data: null
    card: {}
    language: en
    response_type: query_answer
    data:
      targets: []
      success:
        - name: Bathroom
          type: area
          id: bathroom
      failed: []
  conversation_id: null

Problematic Raw

init_options:
  start_stage: intent
  end_stage: intent
  input:
    text: what is the temperature in the bathroom
  pipeline: 01gzmc9kyd6gjsg3e79ne8nekd
  conversation_id: null
stage: done
run:
  pipeline: 01gzmc9kyd6gjsg3e79ne8nekd
  language: en
  runner_data:
    stt_binary_handler_id: null
    timeout: 30
events:
  - type: run-start
    data:
      pipeline: 01gzmc9kyd6gjsg3e79ne8nekd
      language: en
      runner_data:
        stt_binary_handler_id: null
        timeout: 30
    timestamp: "2023-05-05T08:04:14.368342+00:00"
  - type: intent-start
    data:
      engine: homeassistant
      language: en
      intent_input: what is the temperature in the bathroom
    timestamp: "2023-05-05T08:04:14.368798+00:00"
  - type: intent-end
    data:
      intent_output:
        response:
          speech:
            plain:
              speech: Temperature is
              extra_data: null
          card: {}
          language: en
          response_type: query_answer
          data:
            targets: []
            success:
              - name: Bathroom
                type: area
                id: bathroom
            failed: []
        conversation_id: null
    timestamp: "2023-05-05T08:04:14.405445+00:00"
  - type: run-end
    data: null
    timestamp: "2023-05-05T08:04:14.405673+00:00"
intent:
  engine: homeassistant
  language: en
  intent_input: what is the temperature in the bathroom
  done: true
  intent_output:
    response:
      speech:
        plain:
          speech: Temperature is
          extra_data: null
      card: {}
      language: en
      response_type: query_answer
      data:
        targets: []
        success:
          - name: Bathroom
            type: area
            id: bathroom
        failed: []
    conversation_id: null

Successful language processing

intent_output:
  response:
    speech:
      plain:
        speech: Bathroom temperature is 17.9 °C
        extra_data: null
    card: {}
    language: en
    response_type: query_answer
    data:
      targets: []
      success:
        - name: Bathroom Temperature
          type: entity
          id: sensor.temphumid_bathroom_temperature
      failed: []
  conversation_id: null
OzGav commented 1 year ago

I am seeing similar:

What is the temperature in abigail's bedroom
Sorry, I couldn't understand that
What is Abigail's bedroom temperature?
Abigail's bedroom temperature is 17.34 °C

However, my raw shows an intent match failure:

stage: done
run:
  pipeline: 01gzhtprzzc4rkgr276g0ykwr4
  language: en
  runner_data:
    stt_binary_handler_id: null
    timeout: 30
events:
  - type: run-start
    data:
      pipeline: 01gzhtprzzc4rkgr276g0ykwr4
      language: en
      runner_data:
        stt_binary_handler_id: null
        timeout: 30
    timestamp: "2023-05-08T23:01:10.691293+00:00"
  - type: intent-start
    data:
      engine: homeassistant
      language: en
      intent_input: What is the temperature in abigail's bedroom
    timestamp: "2023-05-08T23:01:10.691638+00:00"
  - type: intent-end
    data:
      intent_output:
        response:
          speech:
            plain:
              speech: Sorry, I couldn't understand that
              extra_data: null
          card: {}
          language: en
          response_type: error
          data:
            code: no_intent_match
        conversation_id: null
    timestamp: "2023-05-08T23:01:10.797547+00:00"
  - type: run-end
    data: null
    timestamp: "2023-05-08T23:01:10.797721+00:00"
intent:
  engine: homeassistant
  language: en
  intent_input: What is the temperature in abigail's bedroom
  done: true
  intent_output:
    response:
      speech:
        plain:
          speech: Sorry, I couldn't understand that
          extra_data: null
      card: {}
      language: en
      response_type: error
      data:
        code: no_intent_match
    conversation_id: null
blacknell commented 1 year ago

Anyone else seeing this or understanding something wrong here?

OzGav commented 1 year ago

I’m assuming the devs will pick this up when they have time

blacknell commented 1 year ago

I'm getting new (not previously noticed) behaviour.

Both ensuite and workshop are areas and each has a temperature sensor sensor.temphumid_ensuite_temperature & sensor.temphumid_workshop_temperature

what is the temperature in the ensuite returns Temperature is 18.3 °C what is the temperature in the workshop returns Temperature is

Looking at states I noticed the only difference is that the workshop sensor has a friendly name Workshop and the ensuite sensor is Temperature. Changing the friendly name of the workshop sensor to Temperature removes the erroneous behaviour.

This seems to me wrong. The intent logic should be using the device class and area to work out which one.

Make sense?

tetele commented 1 year ago

This seems to me wrong. The intent logic should be using the device class and area to work out which one. Make sense?

Yes. Work in progress. In the meantime, what you get is a match for "what is <name> in <area>", which doesn't care about device_class at all.

Also, HassClimateGetTemperature intents are not implemented in Home Assistant yet.

blacknell commented 1 year ago

This seems to me wrong. The intent logic should be using the device class and area to work out which one. Make sense?

Yes. Work in progress. In the meantime, what you get is a match for "what is <name> in <area>", which doesn't care about device_class at all.

When you say <name> you're referring to friendly name?

Where are things like work in progress defined/discussed?

tetele commented 1 year ago

When you say you're referring to friendly name?

Or aliases. That's how it works.

Where are things like work in progress defined/discussed?

Too many places, perhaps. There are the intents pull requests and discussions sections, there's the discord channel (dev talks only). For device_class + area for sensors in particular, I've personally been working on something I haven't yet pushed, as it's based on a PR that got merged in a dependency, but hasn't been published to PyPi and updated as such in intents.

brewston commented 11 months ago

@blacknell @OzGav can you test if "What is the temp in [area]" works for you ? see the issue I just opened. Thanks.

OzGav commented 11 months ago

@brewston Unfortunately that does not work for me as only climate entities are supported. I think I recall Paulus saying on a webcast that they were working on how to allow other sensors to feed into this question (all mine are MQTT from either Xiaomi sensors or my Aircon)

blacknell commented 11 months ago

Yes. The challenge is knowing which sensor entity to use. You could have a room sensor and also a computer CPU sensor in same room, for example.

OzGav commented 11 months ago

Yes that’s my understanding of the problem.