Nest Thermostat-1.4 not reporting eco in `preset_mode` when house is set to `Away`

Intecpsp commented 1 year ago

The problem

When setting my Nest thermostat to hold at eco, in the preset_mode attribute, I'm seeing "eco". But when the home is set to away (with the setting of turning the thermostats to eco when away, enabled), the only thing in the data I'm seeing that will alert me to it being eco, is the temperature being way lower than I set it at. The preset_mode attribute is saying none. Is this correct?

What version of Home Assistant Core has the issue?

2022.11.5

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Google Nest

Link to integration documentation on our website

https://www.home-assistant.io/integrations/nest

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 1 year ago

Hey there @allenporter, mind taking a look at this issue as it has been labeled with an integration (nest) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `nest` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign nest` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

nest documentation nest source _{^{(message by IssueLinks)}}

VNRARA commented 1 year ago

It actually seems like it NEVER shows eco when using eco at all.

Nardol commented 1 year ago

I also observe this issue, when my Nest thermostat E is set to echo the preset mode is not changed, in Home Assistant it is still set to eating when the Nest Android app shows the echo mode is enabled.

I observe it on Home Assistant 2022.12.7, core installation.

allenporter commented 1 year ago

Hi, I have not been able to reproduce this (e.g. I set my termostat to home in the nest app, and it is reflected correctly in Home Assistant), so let's talk through steps to get some additional information.

First, some background. The way the nest integration works is:

The nest state is queried when the integration starts or home assistant starts. This includes Nest SDM API traits like eco mode, etc.
Home Assistant starts a pub/sub subscriber
Nest publishes updates to the pub/sub feed with updates to the traits
Home Assistant subscriber receives the updates and applies the changes locally

When filling out the issue template diagnostics are missing. In order to diagnose the issue there are a few things to do:

Enable debugging as described in the Troubleshooting at the bottom
Take diagnostics of the device when it is in eco mode, but not reflected in home assistant
When thermostat is in eco mode, reload the integration. It may or may not enter preset mode, and capturing the diagnostics is interesting
Changing around eco mode on/off should publish messages and you can follow the troubleshooting instructions to see if they are published (console has a way to pull messages), or if they are received (in home assistant logs)

If we have the above information, this may help us track down if there is in fact a home assistant bug (unlikely) or an issue with nest publishing messages or something else. (I am pretty sure this is not a home assistant bug, but likely the thermostat is not publishing the correct messages, which is unfortunate and would mean we'd need to report to the nest API team. However, that is hard if i can't reproduce this.)

allenporter commented 1 year ago

@VNRARA @Nardol you'll need to provide additional details as described above. Eco mode definitely works, but there is a reason why its not for you.

Also, you may want to see what the Google Home App says.

allenporter commented 1 year ago

Also we previously saw a similar report here https://github.com/home-assistant/core/issues/64125 and the Google Home had a weird behavior

Intecpsp commented 1 year ago

We're going through a freeze where I live (7F outside with wind chill down to -14F), so I won't be able to play with the eco settings for a bit. I'll leave this open until I can and let you know when I can adjust those settings and what the outcome(s) are.

Sleashe commented 1 year ago

Experiencing same issue. Will try to troubleshoot and report things in the few weeks.

Triggering eco mode from the app reflects changes in HA integration. But playing with home/away, does not trigger eco mode in HA.

Update (24-01): Temperature isn't updated to HA when triggering away mode from Nest app. Restarting HA doesn't change anything.

Sleashe commented 1 year ago

@allenporter, I did some investigation, enabling advanced logs. Here's what subscription message HA receive when switching from Home to Away in Nest App :

2023-01-24 18:01:59.649 DEBUG (MainThread) [google_nest_sdm.event] EventMessage raw_data={'eventId': '072219ac-9b08-4fec-84b5-e775204b6dd2', 'timestamp': '2023-01-24T17:01:51.524Z', 'resourceUpdate': {'name': 'enterprises/*hidden-id-A*/devices/*hidden-id-B*', 'traits': {'sdm.devices.traits.ThermostatMode': {'mode': 'HEAT', 'availableModes': ['HEAT', 'OFF']}, 'sdm.devices.traits.ThermostatEco': {'availableModes': ['OFF', 'MANUAL_ECO'], 'mode': 'OFF', 'heatCelsius': 16.82999, 'coolCelsius': 24.44444}, 'sdm.devices.traits.ThermostatTemperatureSetpoint': {'heatCelsius': 18.54935}}}, 'userId': 'xxx', 'resourceGroup': ['enterprises/*hidden-id-A*/devices/*hidden-id-B*']}

2023-01-24 18:01:59.649 DEBUG (MainThread) [google_nest_sdm.device] Processing update 072219ac-9b08-4fec-84b5-e775204b6dd2 @ 2023-01-24 17:01:51.524000+00:00

2023-01-24 18:01:59.650 DEBUG (MainThread) [google_nest_sdm.device] Trait update {'sdm.devices.traits.ThermostatMode': <google_nest_sdm.thermostat_traits.ThermostatModeTrait object at 0x7f104bbac700>, 'sdm.devices.traits.ThermostatEco': <google_nest_sdm.thermostat_traits.ThermostatEcoTrait object at 0x7f104bbad150>, 'sdm.devices.traits.ThermostatTemperatureSetpoint':

and here's one trace after manually switching to Eco Mode (where changes are reflected inside HA) :

2023-01-24 18:11:16.080 DEBUG (MainThread) [google_nest_sdm.event] EventMessage raw_data={'eventId': '1e94536d-50e5-4377-8483-ca2f06d05985', 'timestamp': '2023-01-24T17:11:12.310Z', 'resourceUpdate': {'name': 'enterprises/*hidden-id-A*/devices/*hidden-id-B*', 'traits': {'sdm.devices.traits.ThermostatMode': {'mode': 'HEAT', 'availableModes': ['HEAT', 'OFF']}, 'sdm.devices.traits.ThermostatEco': {'availableModes': ['OFF', 'MANUAL_ECO'], 'mode': 'MANUAL_ECO', 'heatCelsius': 16.82999, 'coolCelsius': 24.44444}, 'sdm.devices.traits.ThermostatTemperatureSetpoint': {}}}, 'userId': 'xxx', 'resourceGroup': ['enterprises/*hidden-id-A*/devices/*hidden-id-B*']}

2023-01-24 18:11:16.080 DEBUG (MainThread) [google_nest_sdm.device] Processing update 1e94536d-50e5-4377-8483-ca2f06d05985 @ 2023-01-24 17:11:12.310000+00:00

2023-01-24 18:11:16.080 DEBUG (MainThread) [google_nest_sdm.device] Trait update {'sdm.devices.traits.ThermostatMode': <google_nest_sdm.thermostat_traits.ThermostatModeTrait object at 0x7f104ff44d30>, 'sdm.devices.traits.ThermostatEco': <google_nest_sdm.thermostat_traits.ThermostatEcoTrait object at 0x7f104ff461a0>, 'sdm.devices.traits.ThermostatTemperatureSetpoint': <google_nest_sdm.thermostat_traits.ThermostatTemperatureSetpointTrait object at 0x7f104ff45c00>}

We clearly see Trait Update log is triggered in the first part indicating HA subscribing endpoint is working, but data sent seems to be wrong. While the second trace log clearly indicates that we are switching to MANUAL_ECO. I'm started to think that Google APIs are not returning correct data, or maybe we (@Intecpsp) have misconfigured something ?

allenporter commented 1 year ago

@Sleashe Thank you for confirming that the events appear to be published incorrectly.

I've had some success in reaching nest engineers to investigate if i can get you to put your unredacted data into a private document that can be shared with them (e.g. your google account, device identifiers, serial numbers, and timestamps). I have an open line of communication with some of them and can facilitate, though generally I realize this is weird as you don't typically share PII with random people who maintain open source libraries. I work at Google, but not on Nest. (aporter@google.com)

Sleashe commented 1 year ago

@allenporter no problem. I had some time to investigate and test these last few days. Thanks for reaching Nest engineers, that is super cool.

No problem for sharing personal information with Google Team. Keep me in touch if you need anything else.

Hope they'll find the issue ! 🤞

allenporter commented 1 year ago

To clarify: if you want me to do this I'd need you to make a document with the detail, unredacted logs, and share it with me (e.g At the email above)

VNRARA commented 1 year ago

@VNRARA @Nardol you'll need to provide additional details as described above. Eco mode definitely works, but there is a reason why its not for you.

Also, you may want to see what the Google Home App says.

I can't get the REST to work in postman and I'm losing my shit . I'm not spending more time until you can tell me what to do exactly to get you data to fix this. But I'm believing that the integration doesn't even ask or handle the preset mode correctly. (Srry if I seem a bit angry).

The video shows: I can't get out of ECO mode without setting heating mode to off first. Setting the preset to NONE doesn't remove ECO mode (and resume normale operation). Setting anything else (like temps) sets preset_mode to none (due to it refreshing I believe) but doesn't update that to the API or sets it back to ECO in the list.

https://user-images.githubusercontent.com/3942672/217555312-ea21e9ea-e88c-41cc-97f0-301e2681ebd6.mp4

allenporter commented 1 year ago

@VNRARA Sounds very frustrating. Just to be clear, i have no idea if i can fix this since as far as I can tell this is not a bug in home assistant but in the nest API, and its specific to your device. The data I am looking for is diagnostics and debug logs as I asked for here: https://github.com/home-assistant/core/issues/83312#issuecomment-1358244152 -- basically we need to I cannot reproduce this myself (eco mode works great for me and I can go in and out just fine, and it works fine for most users) so I can either say (1) too bad you're on your own or (2) offer to help prepare low level data for a request to Google to take a look. You don't have to do anything you don't want to do. I acknowledge your frustration.

VNRARA commented 1 year ago

Can you make a similar screen recording showing how you get HA to change modes from and too ECO? I have an EU (heating only) thermostat btw. Reason I actually wanna try Postmanning is because I'm starting to think that the integration OR the API handles EU devices differently.

[Editing as I go]. Added diagnostic file

nest-9f5a51d13d2d9e5d124dcf667314ac81-Woonkamer-0643500209e58879475e80f3733a027f.json (1).txt

Reloading the int in eco mode, didn't do anything. I can deduct by the temp that it's in eco but otherwise no clue.

Is the diag file supposed to change? Because I tried another integration (tuya) and it shows me different states in the file. This one just stays the same with whatever setting I put.

allenporter commented 1 year ago

This is what I did: https://user-images.githubusercontent.com/6026418/217718778-363889c7-afff-48fe-8019-7a231e26ccb2.mov

allenporter commented 1 year ago

@VNRARA When I look at your diagnostics this is what i see:

        "sdm.devices.traits.ThermostatMode": {
          "mode": "HEAT",
          "availableModes": [
            "HEAT",
            "OFF"
          ]
        },
        "sdm.devices.traits.ThermostatEco": {
          "availableModes": [
            "OFF",
            "MANUAL_ECO"
          ],
          "mode": "MANUAL_ECO",
          "heatCelsius": 14.94444,
          "coolCelsius": 24.44443
        },

That shows the thermostat is currently in eco mode. I ran your UI into a translator and it says that "Voorinstelling green" means "Preset green" which I assume means eco mode. Is this a bad translation?

VNRARA commented 1 year ago

Geen = none Eco = eco aka MANUAL_ECO

I downloaded another set of diag file during normal operation (preset none + hvac_mode; heat) and eco:

ECO-nest-9f5a51d13d2d9e5d124dcf667314ac81-Woonkamer-0643500209e58879475e80f3733a027f.json.1.txt

HEAT-config_entry-nest-442eaecbfc5fceb199361069bd991d68.json.txt

Which I guess proves that my HA instance IS getting the correct states, but either not in time or somehow it doesn't get translated correctly to the frontend.

If I change the setting to ECO and download a diag file directly; it's still at OFF. I think something is not updating enough.

Should-BE-ECOconfig_entry-nest-442eaecbfc5fceb199361069bd991d68.json (3).txt

After a reload I get ECO: config_entry-nest-442eaecbfc5fceb199361069bd991d68.json (3).txt

But then the front-end still doesn't update the preset_mode.

allenporter commented 1 year ago

Yes, it implies the pubsub messages are not being delivered by Google or they are being dropped. Having debug logs showing pubsub messages received or not received could be a next step.

allenporter commented 1 year ago

Oh wait so the reload shows the right diagnostics but the UI is wrong? Can you show me the entity state in home assistant developer tools?

Intecpsp commented 1 year ago

@allenporter I just recorded this.

Order of events:

Google Home app reports Home and in Comfort/normal setting
Tell the upstairs thermostat to Hold Eco in Google Home app
- HA reports the temperature and preset_mode changes
End hold in Google Home app (took a sec to fully flip back)
- HA reports the temperature and preset_mode changes
Set home to Away in Google Home app
- HA reports the temperature change, but not preset_mode when the Google Home app reports Eco for both thermostats
Set back to Home in Google Home app
- HA reports the temperature change

https://user-images.githubusercontent.com/1164012/217856471-d4a9cfcc-d522-452d-8be5-ea5af781fad2.mov

Google Home app after manually setting to Hold Eco Screenshot_20230209-102439

Google Home app after setting Away (you can see eco in the background, but it wasn't manually set, so it's asking if I want to set it) Screenshot_20230209-102505

allenporter commented 1 year ago

So you're saying the issue is:

Home is set to away
Google puts the thermostat in eco mode
No eco mode messages are published

@Intecpsp if you reload HA, it will fetch new information from the API w/o event messages published. I'm curious what preset_mode looks like in that case?

VNRARA commented 1 year ago

I just deleted a whole piece of shit yaml from customize.yaml concerning the climate and considering a specific work device tracker to be home and ECO works properly now... what the actual bleep

climate.woonkamer:
  hvac_modes:
    - heat
    - "off"
  min_temp: 7
  max_temp: 32
  preset_modes:
    - none
    - eco
  hvac_action: heating
  preset_mode: none
  friendly_name: Woonkamer
  supported_features: 17
  device_tracker.dhlpc:
  consider_home: 1
  track: true

Intecpsp commented 1 year ago

So you're saying the issue is:

Home is set to away

Google puts the thermostat in eco mode

No eco mode messages are published

@Intecpsp if you reload HA, it will fetch new information from the API w/o event messages published. I'm curious what preset_mode looks like in that case?

"Reload" = restart?

allenporter commented 1 year ago

@Intecpsp Under Settings > Devices & Services on the Integrations tab find the Google Nest integration and in the vertical ... menu of the integration you can select Reload. This will unload the entities and config, then restart it which will query the API to get initial device state and restart the pubsub subscriber.

allenporter commented 1 year ago

@VNRARA great, i don't follow specifically since i'm not familiar with that yaml configuration but sounds like things are working for you.

Intecpsp commented 1 year ago

@Intecpsp Under Settings > Devices & Services on the Integrations tab find the Google Nest integration and in the vertical ... menu of the integration you can select Reload. This will unload the entities and config, then restart it which will query the API to get initial device state and restart the pubsub subscriber.

Reloading doesn't pull the correct preset_mode

Nardol commented 1 year ago

I should make more tests on my side but when I tested, when I restarted Home Assistant completely preset mode was correct.

VNRARA commented 1 year ago

@VNRARA great, i don't follow specifically since i'm not familiar with that yaml configuration but sounds like things are working for you.

customize.yaml can be used to do overrides on certain aspects of entities. I have no clue how it got in my stuff, but I'd recommend @Intecpsp to check that file as well.

Intecpsp commented 1 year ago

I should make more tests on my side but when I tested, when I restarted Home Assistant completely preset mode was correct.

Just tried a full HA restart and did not see the expected preset_mode for me sadly

allenporter commented 1 year ago

@Intecpsp and @Nardol just to be clarify: 1) is this specific to Away 2) when manually setting eco?

It might also be useful to know if this only happens to specific thermostat "Model" and "Software" version. For example in the Nest app my model is Display-4.8, Backplate-7.6 with Software 6.2-27. My suspicious is that the issue is on the cloud side, but could be how specific devices are registered.

allenporter commented 1 year ago

@Intecpsp OK one thing I've noticed is that eco mode only has OFF or MANUAL_ECO. It does not have a mode for automatic eco. I was perhaps wondering if this was intentional by the API designers... Notably, they removed home / away from the API as a first class object i wonder if this is a privacy restriction.

Intecpsp commented 1 year ago

@Intecpsp and @Nardol just to be clarify:

is this specific to Away

when manually setting eco?

It might also be useful to know if this only happens to specific thermostat "Model" and "Software" version. For example in the Nest app my model is Display-4.8, Backplate-7.6 with Software 6.2-27. My suspicious is that the issue is on the cloud side, but could be how specific devices are registered.

Manually setting eco seems to be working. Setting to Away isn't behaving how I'd expect it to.

(both of mine are the same model) Model: Thermostat-1.4 Software version: 1.1-11

Intecpsp commented 1 year ago

@Intecpsp OK one thing I've noticed is that eco mode only has OFF or MANUAL_ECO. It does not have a mode for automatic eco. I was perhaps wondering if this was intentional by the API designers... Notably, they removed home / away from the API as a first class object i wonder if this is a privacy restriction.

Dang, so are you saying there is no way to set preset_mode when setting the home to Away because you won't know when it actually goes to Away? I guess I'll stick to my temperature check as a workaround then.

VNRARA commented 1 year ago

Home/away = a Google Home thing. Not a nest trait (yet). Correct?

If you can set the thermostat via Google Home or Nest to ECO and it shows correctly, then the integration is working as expected for the current API, right?

@Intecpsp Btw Are you using Google Home preview or something? My app doesn't have the eco hold with comfort mode and sleep and end time like yours has. I think you have the Mirrored Thermostat 2020 right? I have the v3.

Intecpsp commented 1 year ago

I have the latest Nest device (with the touch-sensitive right side) that is only able to be accessed through the Google Home app. I've had previous revisions that were in the Nest app. Things are a lot different in the Google Home app for first-party (non-legacy) devices.

VNRARA commented 1 year ago

Still no clue which one that is.

Intecpsp commented 1 year ago

This one: https://store.google.com/us/product/nest_thermostat

https://store.google.com/us/magazine/compare_thermostats

allenporter commented 1 year ago

I wonder if you can use a home/away routine to explicitly change the thermostat mode or if it's a "no op" since it automatically sets eco

VNRARA commented 1 year ago

@Intecpsp US seems to have the Nest Thermostat (mirror and touchside) and Nest Learning Therm. (v1,2 and 3) EU has Nest Thermostat E (non-learning) and Nest Learning Therm V3.

The API doesn't accommodate the full possibilities of the cheaper and newer Nest Thermostat which you pointed out. Only the ones with the rounded leaf for eco. Those are the Nest E and Learning thermostat.

They've only added "manual_eco" and "off' to the API and not the other modes that the Nest Thermostat (mirror and touchside) has. @allenporter can you confirm this?

Nardol commented 1 year ago

I tested again and now even with a full HA restart it does not change to the right mode. I might have missed something last time I noticed that.

Setting manually the thermostat to Eco from Google Home make the change in Home Assistant as expected, it is only when away mode is switched. Same if it is automatically when my phone is detected as away or if I change mode manually from the Google Home app.

In the Nest app I have:

Model: Nest Thermostat E 1.1
Support model: Backplate8.2
Software: 6.1-9

It is really at home/away modes in Google Home which change thermostat mode to away but not in Home Assistant.

Sleashe commented 1 year ago

Setting manually the thermostat to Eco from Google Home make the change in Home Assistant as expected, it is only when away mode is switched. Same if it is automatically when my phone is detected as away or if I change mode manually from the Google Home app.

@allenporter it's the same issue/behaviour I'm experiencing ! Meaning I'm not the only one ! :)

Intecpsp commented 1 year ago

I wonder if you can use a home/away routine to explicitly change the thermostat mode or if it's a "no op" since it automatically sets eco

Looks like it's using those for this purpose (I don't use Google Routines since I have everything in HA, so these are default)

Screenshot_20230209-130854 Screenshot_20230209-130909

Sleashe commented 1 year ago

Hey @allenporter !

My Nest thermostat became unavailable this night. Trying to setup the integration again but my entities entirely disappeared. Do you think guys on the Nest team are doing something with my account ?

I'm experiencing this : Issue 80675

I'm logging these kind of response from the API : 2023-02-16 13:57:29.518 DEBUG (MainThread) [google_nest_sdm.auth] response={'devices': [{'name': 'enterprises/[enterprise-id]/devices/[device-id]', 'type': 'sdm.devices.types.THERMOSTAT', 'assignee': 'enterprises/c8b2b77e-dc8a-4102-b131-d112344e1d03/structures/[structure-id]/rooms/[room-id]', 'traits': {}, 'parentRelations': [{'parent': 'enterprises/c8b2b77e-dc8a-4102-b131-d112344e1d03/structures/[structure-id]/rooms/[room-id], 'displayName': 'Dining Room'}]}]}

Any idea ? Thanks a lot in advance !

allenporter commented 1 year ago

@Sleashe please don't report support issues on unrelated issues that already exist. This issue is about a specific bug. Also there are already 3-4 other issues filed related to a nest outage. This is not a home assistant bug.

Sleashe commented 1 year ago

@Sleashe please don't report support issues on unrelated issues that already exist. This issue is about a specific bug. Also there are already 3-4 other issues filed related to a nest outage. This is not a home assistant bug.

My apologies, I posted the message just before seeing there was other topics about this issue.

jbrody17 commented 1 year ago

I have also been experiencing this issue. My testing has revealed that when manually setting my thermostats to ECO mode, the status is correctly reported to HA. However, when Away mode is triggered (and Eco mode is activated), the status is not reported to HA. Same behavior whether using Nest app or Google Home app.

My temporary workaround has been to use the Google Home integration and enable/disable an input_boolean helper based on the home status. Only thing I dislike is that this is home-wide status, not thermostat specific status. Also this is one way communication (Google telling HA the sensors are away). I'd prefer to have 2 way communication (HA telling Google it should be away).

There was a thread about this as well.

allenporter commented 1 year ago

As I was saying above https://github.com/home-assistant/core/issues/83312#issuecomment-1424436513 I don't think this is an issue. I think ya'll are asking for a feature request from the nest API that doesn't exist.

allenporter commented 1 year ago

See https://developers.google.com/nest/device-access/traits/device/thermostat-eco#fields for what is supported by the API. It supports OFF or MANUAL_ECO i think the point may be they 3rd parties abusing the API to stalk someone.

allenporter commented 1 year ago

I am closing this issue since it is not a feature exposed by the nest SDM APIs. If anything new related to this is exposed to the API, would be happy to consider a feature request in the community forums (happy to implement everything possible exposed from the nest SDM API)

Many people are interested in eco mode to use it as presence detection. My impression is that best way to do this is with Google Assistant integration with Home Assistant. My impression is that this is a much more direct way to do presence detection, since it is a first class feature of the assistant API. (Using "eco mode" is pretty weird, right?). Here is an example thread from the community forum but I haven't tried this myself.

This won't help you get a correct thermostat mode if you really do need to know that the thermostat is in eco mode, but perhaps it will give you something to work with since you can assume eco mode may be set when away.

I'd encourage folks to discuss in the community forum if there are additional topics here.

home-assistant / core