Honeywell Lyric integration device frequently unavailable

[michaelsleen]

The problem

My thermostat device in the Honeywell Lyric integration frequently shows as "unavailable." Reloading the integration immediately resolves this temporarily, but the issue soon repeat itself. I estimate this began sometime in the past week.

What version of Home Assistant Core has the issue?

core-2024.1.6

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Honeywell Lyric

Link to integration documentation on our website

https://www.home-assistant.io/integrations/lyric

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @timmo001, mind taking a look at this issue as it has been labeled with an integration (lyric) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `lyric` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign lyric` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

lyric documentation lyric source _{^{(message by IssueLinks)}}

[guillaumeharvey26]

Same issue here!

[mefbgil]

I'm getting "Timeout fetching lyric_coordinator data" sporadically since Feb 4th. Running latest docker 2024.1.6 here.

[johanmartinus]

Same here, since this week?

[guillaumeharvey26]

Same here, since this week?

Yes I have many "Timeout fetching lyric_coordinator data" since Feb 4th. Version 2024.1.6

[TheGroundZero]

Same problem here. Reloading the integration helps temporarily.

Logger: homeassistant.components.lyric
Source: helpers/update_coordinator.py:306
Integration: Honeywell Lyric (documentation, issues)
First occurred: 5 February 2024 at 14:10:04 (29 occurrences)
Last logged: 16:59:04

Timeout fetching lyric_coordinator data

[tmikoss]

My 2 cents: it seems to be something related to API capacities on Honeywell end, since the unavailability was happening a lot during the firs week of February, but now has dropped to once or twice a day, without me doing anything to the HA instance.

Also, in the original screenshot, and also in my case, unavailability lasts for exactly 5 minutes - is that a the default retry timeout?

[swman77]

I have to reload the integration evrey time if I want to see the current status. core HA 2024.3.1 OS HA 2.1

[Chef-de-IT]

Same issue here; also started relatively recently (within units of weeks ago). From the log:

Logger: homeassistant.components.lyric
Source: helpers/update_coordinator.py:328
integration: Honeywell Lyric (documentation, issues)
First occurred: April 20, 2024 at 6:09:27 PM (29 occurrences)
Last logged: 7:38:34 PM

Error requesting lyric_coordinator data: Cannot connect to host api.honeywell.com:443 ssl:default [Try again]

[michaelsleen]

This issue has flared up again. Is there a fix?

[jasstrong]

I have tried every May and every June release so far of HomeAssistant and none of them have worked with the Honeywell Lyric integration. I have to revert to 2024.4.4 which I fortunately have a backup of.

Attached is a redacted screenshot of the error.

[Chef-de-IT]

I have tried every May and every June release so far of HomeAssistant and none of them have worked with the Honeywell Lyric integration. I have to revert to 2024.4.4 which I fortunately have a backup of.

The HA instances of mine that have Lyric are on 2024.4.4 but all are production ones for 3rd parties. I wish I could test your experience on newer core versions - but no Lyric in the lab. It's difficult to imagine the newer HA core version would alter the Lyric API URLs to result in a 400 but ofc. not impossible.

Curiously, 400 is supposed to be "Bad Request" - not "Device not found" which I surmise ought to be 404, but it's up to the Honeywell API server-side devs to explicitly set those up for the errors as they see fit - so they could technically use whatever return code they please. Maybe they'd reserved the 404 for when the endpoint itself isn't found but when the endoiint works but your thermostat isn't found in their database, then they return 400 to differentiate and have a DB team ignore it as opposed to having the engineering team ignore it as with err 500 - or having the capacity management / operations team ignore it if it's err 503 below or no response from the ednpoint at all. For a mature and structured company like Honeywell it must be really important that the customer's issue is being routed to and ignored by the appropriate team. I mean, every one of the errors, yours & mine, are visible in the logs on their end also.

On 2024.4.4 from time to time I'm getting a couple different error types vs. yours:

'faultstring': 'The Service is temporarily unavailable', 'detail': {'errorcode': 'messaging.adaptors.http.flow.ServiceUnavailable'}}}, 'status': 503} ... (together with other errors we observe as captured above) lays bare that Honeywell's cloud API endpoint https://api.honeywell.com/v2/ is hitting server-side capacity limits at multiple points in their stack.

On my 2024.4.4 HA, the above errors are recoverable. If they go on for long enough, HA will disable the Lyric integration, but, this being HA, I have a script that detects it and tries to reload it a couple of times, after 20 min and after 2hrs, as well as every morning if it's in the disabled state. This kludge is as ugly as it is effective for the time being, and I think it'll outlive my best work.

This is a lesson for us all to avoid internet and cloud reliance in our HA devices like a plague, because IMHO almost EVERY cloud API will eventually be where https://api.honeywell.com/v2/ is today - it's just a matter of when. However, with IP thermostats, there are hardly any practical alternatives. I have one Proliphix thermostat from years ago which is PoE wired and local, but they're no longer available new.

[jasstrong]

Having just installed, and then immediately rolled back, 2024.6.3, with the exact same consequences, what information would you like me to extract? It's easy enough to install the updated core and easy enough to roll it back, so if there's something I can do to help debug this, please let me know.