Nest integration fails during setup with 504 Deadline Exceeded

manningmichael2 commented 1 month ago

The problem

Logger: homeassistant.components.nest.config_flow Source: components/nest/config_flow.py:287 integration: Google Nest (documentation, issues) First occurred: 9:09:20 AM (17 occurrences) Last logged: 10:59:41 AM

Error creating subscription: Failed to create subscription 'projects/PROJECTID/subscriptions/home-assistant-MYQ71YQ7LR': 504 Deadline Exceeded

What version of Home Assistant Core has the issue?

2024.10.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Nest

Link to integration documentation on our website

https://www.home-assistant.io/integrations/nest

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: homeassistant.components.nest.config_flow
Source: components/nest/config_flow.py:287
integration: Google Nest (documentation, issues)
First occurred: 9:09:20 AM (17 occurrences)
Last logged: 10:59:41 AM

Error creating subscription: Failed to create subscription 'projects/PROJECTID/subscriptions/home-assistant-MYQ71YQ7LR': 504 Deadline Exceeded
Error creating subscription: Failed to create subscription 'projects/PROJECTID/subscriptions/home-assistant-0DfIzRKIu0': 504 Deadline Exceeded
Error creating subscription: Failed to create subscription 'projects/PROJECTID/subscriptions/home-assistant-qQ5GTJeafL': 504 Deadline Exceeded
Error creating subscription: Failed to create subscription 'projects/PROJECTID/subscriptions/home-assistant-QCJIL2rxfN': 504 Deadline Exceeded
Error creating subscription: Failed to create subscription 'projects/PROJECTID/subscriptions/home-assistant-O1PwM53aTR': 504 Deadline Exceeded

Additional information

new install of home assistant on HA Green. Integration previously worked on this device, but I dont know which version it was running. May have failed when I updated.

home-assistant[bot] commented 1 month ago

Hey there @allenporter, mind taking a look at this issue as it has been labeled with an integration (nest) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `nest` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign nest` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

nest documentation nest source _{^{(message by IssueLinks)}}

allenporter commented 1 month ago

Seems like similar to symptoms in https://github.com/home-assistant/core/issues/124096

Would you be able to turn up debug logs on this?

logger:
  default: info
  logs:
    google_nest_sdm: debug
    google_nest_sdm.google_nest_subscriber: debug

garysargentpersonal commented 1 month ago

@manningmichael2 I had a similar error the other day which led me here, however in my case it turned out to be because the Google calls were routing through a VPN.

I'd routed YouTube through a VPN but if turned out the Nest API shared the same IP addresses as YouTube and called the Nest calls to go over the VPN. Also affected my access to developer.android.com

If you have any VPN routes enabled try disabling them.

StevieAlford commented 1 month ago

Had this same error last night, tried the setup again today and the error was gone. I remember seeing a Google screen during the setup steps saying it can take a few minutes to a few hours for it to complete.

Chobzi commented 2 weeks ago

Hi @allenporter

Completely new to Home Assistant (received and started to set-up home assistant green yesterday) so pardon potential lack of process/ knowledge, but I seem to be experiencing the same/ a similar issue as listed by @manningmichael2.

I followed the very clear instructions as set out in the integration guide.

After following the steps I'm presented with an error message in the HA UI. I noticed that in the first instance I used the (likely wrong) capitalised Project ID Name, instead of the Google Project ID. I changed this but was presented with the same error message.

Based on suggestions above I have waited overnight to retry, but am presented with the same error still.

I have read through #124096 but am not sure whether that is the root cause for me and if so, how I could address this.

Please find error log below.

Log Details (ERROR) Logger: homeassistant.components.nest.config_flow Source: components/nest/config_flow.py:287 integration: Google Nest (documentation, issues) First occurred: 29 October 2024 at 15:19:56 (5 occurrences) Last logged: 10:18:02

Error creating subscription: Failed to create subscription 'projects/HA-Nest-Integration-0000001/subscriptions/home-assistant-KIwrZjwZH7': 504 Deadline Exceeded Error creating subscription: Failed to create subscription 'projects/ha-nest-integration-0000001/subscriptions/home-assistant-67rbMikq7B': 504 Deadline Exceeded Error creating subscription: Failed to create subscription 'projects/ha-nest-integration-0000001/subscriptions/home-assistant-PslIgl7MuV': 504 Deadline Exceeded Error creating subscription: Failed to create subscription 'projects/ha-nest-integration-0000001/subscriptions/home-assistant-xUV5M9AgK9': 504 Deadline Exceeded Error creating subscription: Failed to create subscription 'projects/ha-nest-integration-0000001/subscriptions/home-assistant-kKE41ANJFC': 504 Deadline Exceeded

Please let me know if and how I could contribute to resolution of this.

allenporter commented 2 weeks ago

Thank you for reporting this detail. What would be most helpful is if you can enable debug logging as mentioned in https://github.com/home-assistant/core/issues/128033#issuecomment-2403767427 and get more debug log contents of what is happening during the configuration flow.

For release 2024.11 I have re-written the configuration flow to now take multiple steps because we are changing how topics and subscriptions work and are set up given some upcoming nest changes. This includes the option to manually create topics and subscriptions which is unfortunately more complex for end users, but may give us additional steps where we can manually change something (e.g. you have the option to manually create a subscription and use it rather than having nest create it for you) which may help.

Chobzi commented 2 weeks ago

Hi @allenporter, I found out that the HA version I was running was still 12.1. I just updated the system to 13.1 and retried adding the Nest integration and this now worked on the first try. Assume it was something in 12.1 that has already been fixed in the latest release. Apologies for not checking if my system was on latest version in first instance!

xavier114fch commented 1 week ago

Hi @allenporter - I got the same 504 error similar to the reporter by following the integration guide. I just updated HA Core to 2024.11.0 (Supervisor at 2024.11.2 and OS at 13.2), deleted the previous application credentials and started every steps afresh. After providing the project ID from Device Access Console, there is an error prompt without any messages.

I inspected the log (previously setup to show debug logs from above) and it showed this.

2024-11-06 23:16:17.466 INFO (MainThread) [homeassistant.components.nest.config_flow] Successfully authenticated
2024-11-06 23:16:17.467 DEBUG (MainThread) [google_nest_sdm.auth] request[get]=https://pubsub.googleapis.com/v1/projects/sdm-prod/topics/enterprise-b9c13fb4-8735-493d-81b0-8bc5d607d223
2024-11-06 23:16:17.468 DEBUG (MainThread) [google_nest_sdm.auth] request[get]=https://pubsub.googleapis.com/v1/projects/nest-ha-integration-440923/topics
2024-11-06 23:16:18.206 DEBUG (MainThread) [google_nest_sdm.admin_client] SDM topic exists but we do not have permission to access it (expected)
2024-11-06 23:16:18.277 DEBUG (MainThread) [google_nest_sdm.auth] response={}
2024-11-06 23:16:18.277 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 477, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 559, in _handle
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 83, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 86, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 242, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 81, in with_admin
    return await func(self, request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 215, in get
    return await super().get(request, flow_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 108, in get
    result = await self._flow_mgr.async_configure(flow_id)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 367, in async_configure
    result = await self._async_configure(flow_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 414, in _async_configure
    result = await self._async_handle_step(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 517, in _async_handle_step
    result: _FlowResultT = await getattr(flow, method)(user_input)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/config_entry_oauth2_flow.py", line 361, in async_step_creation
    return await self.async_oauth_create_entry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/nest/config_flow.py", line 133, in async_oauth_create_entry
    return await self.async_step_pubsub()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/nest/config_flow.py", line 261, in async_step_pubsub
    eligible_topics = await self._admin_client.list_eligible_topics(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google_nest_sdm/admin_client.py", line 217, in list_eligible_topics
    (sdm_topic_task, cloud_topics_task) = await asyncio.gather(
                                          ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google_nest_sdm/admin_client.py", line 210, in get_cloud_topics
    return await self.list_topics(f"projects/{self._cloud_project_id}")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google_nest_sdm/admin_client.py", line 141, in list_topics
    return [topic["name"] for topic in response["topics"]]
                                       ~~~~~~~~^^^^^^^^^^
KeyError: 'topics'

It seems to me the authentication process returns an empty object that causes the KeyError.

allenporter commented 1 week ago

@manningmichael2 thank you for the report, but please open a new issue -- we should not track here given the errors are different. this issue is specific to the new home assistant release onboarding flow and seems pretty urgent to resolve.

Edit: That error is now discussed in https://github.com/home-assistant/core/issues/130042

home-assistant / core