Google assistant device sync error 500 following 2023.10 update

[NarkyAbyss]

The problem

Following update to 10.0 and also 10.1 google home fails to sync devices sporadically. No google cloud or service account changes made between updates. Successfully updated from initial 8.3 to 9.3 without issue. restoring to 9.3 not fixing issue.

google cloud metrics reporting high error rate: google.home.graph.v1.HomeGraphApiService.ReportStateAndNotification | 5,360 | 0.04% | 0.325 seconds google.home.graph.v1.HomeGraphApiService.RequestSyncDevices | 11 | 45.45% | 4.698 seconds

What version of Home Assistant Core has the issue?

core-2023.10.1

What was the last working version of Home Assistant Core?

core-2023.9.3

What type of installation are you running?

Home Assistant OS

Integration causing the issue

google_assistant

Link to integration documentation on our website

https://www.home-assistant.io/integrations/google_assistant

Diagnostics information

No response

Example YAML snippet

google_assistant:
  project_id: !secret google_project
  service_account: !include SERVICE_ACCOUNT.JSON
  report_state: true
  expose_by_default: true
#   exposed_domains:
#     camera
#     media_player
#     vacuum
  entity_config:
    camera.baby_cam_cam:
        expose: true
    sensor.processor_temperature:
        expose: false
    media_player.baby_cam_media_player:
        expose: false

Anything in the logs that might be useful for us?

Logger: homeassistant.components.google_assistant.http
Source: components/google_assistant/http.py:217
Integration: Google Assistant (documentation, issues)
First occurred: 8:56:05 PM (1 occurrences)
Last logged: 8:56:05 PM

Request for https://homegraph.googleapis.com/v1/devices:requestSync failed: 500

Additional information

Home Assistant 2023.9.3 Supervisor 2023.10.0 Operating System 10.5 Frontend 20230911.0 - latest

[home-assistant[bot]]

Hey there @home-assistant/cloud, mind taking a look at this issue as it has been labeled with an integration (google_assistant) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `google_assistant` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign google_assistant` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

google_assistant documentation google_assistant source _{^{(message by IssueLinks)}}

[awaldram]

In addition which leads me to believe it may be 'google api related' rather than HA version . Devices go on and offline randomly.

Re-linking generally fails with 'Google-project' unreachable after HA authentication but will occasionally pass. Creating a new project/key pairs manually also fails more often than succeeds.

fails like either timing or api call flow changing.

this seems related

https://community.home-assistant.io/t/can-no-longer-link-google-assistant/409495/73

[Cr4z33]

@awaldram that's me and I confirm the issue is still going on.

At first I thought it was something related to DuckDNS (they often have issues), but the same happens if I try to use CloudFlare or other DNS solutions.

[NarkyAbyss]

In addition which leads me to believe it may be 'google api related' rather than HA version . Devices go on and offline randomly.

Re-linking generally fails with 'Google-project' unreachable after HA authentication but will occasionally pass. Creating a new project/key pairs manually also fails more often than succeeds.

fails like either timing or api call flow changing.

this seems related

https://community.home-assistant.io/t/can-no-longer-link-google-assistant/409495/73

hmm, i noticed my test app is no longer configured

@Cr4z33 did you also notice your test app needs to be redeployed?

https://www.home-assistant.io/integrations/google_assistant#:~:text=Select%20the%20Develop,isn%E2%80%99t%2C%20enable%20it).

[NarkyAbyss]

additional logs from google cloud. Not very descriptive.

{ "insertId": "<redacted>", "jsonPayload": { "syncLog": { "syncs": [ { "status": "RESOURCE_EXHAUSTED", "requestId": "<redacted>", "httpLatencyMsec": "2825" } ] } }, "resource": { "type": "assistant_action_project", "labels": { "project_id": "<redacted>" } }, "timestamp": "2023-10-06T09:43:11.611417703Z", "severity": "ERROR", "logName": "projects/<redacted>/logs/assistant_smarthome%2Fassistant_smarthome_logs", "receiveTimestamp": "2023-10-06T09:43:11.611417703Z" }

{ "insertId": "<redacted>", "jsonPayload": { "errorReason": "Failed to get response from 3P. 3P returned malformed response like invalid response code or un-inflatble body.", "request": { "body": "grant_type=refresh_token&refresh_token=REDACTED_VALUE&client_id=https://oauth-redirect.googleusercontent.com/r/<redacted>&client_secret=REDACTED_VALUE", "method": "POST", "uri": "https://<redacted>/auth/token" }, "step": "REFRESH_ACCESS_TOKEN", "sessionId": <redacted>, "@type": "type.googleapis.com/google.identity.accountlinking.type.AccountLinkingError" }, "resource": { "type": "assistant_action_project", "labels": { "project_id": "<redacted>" } }, "timestamp": "2023-10-07T15:07:57.143Z", "severity": "ERROR", "logName": "projects/<redacted>/logs/accountlinking-pa.googleapis.com%2Ferror", "receiveTimestamp": "2023-10-07T15:07:57.178449627Z" }

{ "insertId": "<redacted>", "jsonPayload": { "syncLog": { "syncs": [ { "requestId": "<redacted>", "status": "GAL_BAD_3P_RESPONSE", "httpLatencyMsec": "2586" } ] } }, "resource": { "type": "assistant_action_project", "labels": { "project_id": "<redacted>" } }, "timestamp": "2023-10-11T18:00:10.763531433Z", "severity": "ERROR", "logName": "projects/<redacted>/logs/assistant_smarthome%2Fassistant_smarthome_logs", "receiveTimestamp": "2023-10-11T18:00:10.763531433Z" }

[Cr4z33]

@NarkyAbyss yeah I already ran the test as I used to do in the past for other projects, but as you might have noticed the issue is still there. 🙁

[JayJay-SmartHome]

Can I check as this is something I am experiencing but also noticed that my Raspberry pi with ha installed keeps dropping "offline" from home network. My access is random as sometimes can access Lovelace and sometimes not. Think the not is due to the port forwarding not happening as the pi is "offline". Be good to know a solution to not going offline

[Cr4z33]

I just wanted to add that with latest Supervisor and OS updates everything is back working on my side!

[servernotifiche]

maybe it's just a coincidence, but the integration with google home crashes immediately after launching a tts.google_say service

[JayJay-SmartHome]

It's working for me too Got it working before the recent updates.

[IgnatievArkadiy]

I have disconnects with Google home (DuckDns), it happens quite often!!! But after I enter App on phone after some time it recovers.

Logger: homeassistant.components.google_assistant.http
Source: components/google_assistant/http.py:217
Integration: Google Assistant (documentation, issues)
First occurred: 21:42:00 (1 occurrences)
Last logged: 21:42:00

Request for https://homegraph.googleapis.com/v1/devices:requestSync failed: 500

i think problem in Cover type device (blinds, windows) become when I said "ok, google open windows" HA disconnects

[Cr4z33]

I have disconnects with Google home (DuckDns), it happens quite often!!! But after I enter App on phone after some time it recovers.

Logger: homeassistant.components.google_assistant.http
Source: components/google_assistant/http.py:217
Integration: Google Assistant (documentation, issues)
First occurred: 21:42:00 (1 occurrences)
Last logged: 21:42:00

Request for https://homegraph.googleapis.com/v1/devices:requestSync failed: 500

i think problem in Cover type device (blinds, windows) become when I said "ok, google open windows" HA disconnects

I am having disconnects too indeed!

At first I was happy with getting everything back working, but then during the day from time to time Google was replying saying he couldn't find Home Assistant v2 (or that it wasn't responding, cannot remember exactly tbh).

[controller999]

my problem still persists, i cannot add the home assistant plug in to google home at all, google appears not to be able to connect to home assistant. home assistant is stating in the logs that the connection has failed due to incorrect authentication. this was happening with either duckdns or nabu casa plugins, i have no problems connecting to home assistant it is just google

[Cr4z33]

@controller999 go to https://console.actions.google.com/u/0 then enter your project and run a test.

It's worth a try.

[controller999]

thank you, tried doing as you suggested but all tests came back as "there was a problem" Nabu casa seems to be working ok though so will probably have to use that and cough up the money

[slovenec88]

controller999 I'm pretty sure that won't help, as Nabu Casa -> Google Assistant is also not working. It's not showing any exposed entities for a few days now.

[controller999]

thanks everyone for all your help, i think i may have fixed the problem. I deleted the original project on google cloud and created another one, seems to be working for now, fingers crossed

[cruux13]

thanks everyone for all your help, i think i may have fixed the problem. I deleted the original project on google cloud and created another one, seems to be working for now, fingers crossed

Mine went back to normal operation two days ago. Did not have to create new project. Probably this was an issue at Google's side.

[wolfjank]

Got the same problem. I also ran a test. Didn’t work

[controller999]

still got a problem with google, one minute its working and connected then it will disconnect for no reason. tried testing and the only thing i got was an upgrade to firebase which i didnt intend. sometimes if i continue to ask google will give in and connect me

[ilBiancoMiglio]

For the past 2 days, I've also been experiencing the same issue with a pretty standard configuration on Home Assistant, having only exposed the lights. After trying everything without success (I've deleted and recreated the project both on Google Actions and GCloud, changed the domain on duckDNS, altered the ports on the router and NGNX), I managed to solve it by removing all the tokens from the HA profile and by disconnecting and reconnecting the action from the Google Home app. I hope this might be helpful to someone! Let me know if this workaround works for anyone!

[Foltifeki]

I have the same or similar problem, it was working for a few weeks and today it doesn't page at all

[SimpleSon]

Just ran into this. I'm going to hold off on chasing this down until I see if it resolves itself with time. Talking to the other APIs just fine it seems.

[fargiolas]

Similar problems here, had to generate a new private key as the old one disappeared from GCP and re-enable homegraph API as it somehow disabled itself. Doubt it's anything on Home Assistant side.

[servernotifiche]

I abandoned duckdns 3 weeks ago and switched to cloudflare tunnel + domain. as regards the problem of "I can't access homeassistant" pronounced by various Google devices when invoking a service, I solved it 100% with Cloudflare tunnel. if I ask google assistant to sync my devices, it does it every time without error.

If I ask HomeAssistant to sync the devices via the dashboard, the result is still this: Invocation of the button/press service failed. "Unable to sync devices with result code: 403, check log for more info."

[issue-triage-workflows[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.