search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
69.75k stars 28.91k forks source link

curl command timeouts for Octopus Energy API #115748

Closed SNCWilkinson closed 2 months ago

SNCWilkinson commented 2 months ago

The problem

Since upgrading the Octopus rate third party addon has stopped working. Looking a debug logs I see the following:

2024-04-17 08:30:25.629 WARNING (MainThread) [custom_components.octopus_energy.api_client] Failed to connect. Timeout of ClientTimeout(total=None, connect=None, sock_read=20, sock_connect=20, ceil_threshold=5) exceeded. 2024-04-17 08:30:25.630 ERROR (MainThread) [aiohttp.server] Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1025, in _wrap_create_connection return await self._loop.create_connection(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1102, in create_connection sock = await self._connect_sock( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1005, in _connect_sock await self.sock_connect(sock, address) File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 641, in sock_connect return await fut ^^^^^^^^^ asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 578, in _request conn = await self._connector.connect( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 544, in connect proto = await self._create_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 944, in _createconnection , proto = await self._create_direct_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1226, in _create_direct_connection transp, proto = await self._wrap_create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1022, in _wrap_create_connection async with ceil_timeout( File "/usr/local/lib/python3.12/asyncio/timeouts.py", line 115, in aexit raise TimeoutError from exc_val TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/octopus_energy/api_client/init.py", line 424, in async_refresh_token async with client.post(url, json=payload) as token_response: File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 1194, in aenter self._resp = await self._coro ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 582, in _request raise ServerTimeoutError( aiohttp.client_exceptions.ServerTimeoutError: Connection timeout to host https://api.octopus.energy/v1/graphql/

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request resp = await request_handler(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 543, in _handle resp = await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 114, in impl return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 83, in forwarded_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 88, in ban_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware response = await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle result = await handler(request, request.match_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 71, in with_admin return await func(self, request, *args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 285, in post return await super().post(request, flow_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 73, in wrapper result = await method(view, request, data, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 122, in post result = await self._flow_mgr.async_configure(flow_id, data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 356, in async_configure result = await self._async_configure(flow_id, user_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 402, in _async_configure result = await self._async_handle_step( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 506, in _async_handle_step result: _FlowResultT = await getattr(flow, method)(user_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/octopus_energy/config_flow.py", line 537, in async_step_user errors = await async_validate_main_config(config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/octopus_energy/config/main.py", line 72, in async_validate_main_config account_info = await client.async_get_account(data[CONFIG_ACCOUNT_ID]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/octopus_energy/api_client/init.py", line 442, in async_get_account await self.async_refresh_token() File "/config/custom_components/octopus_energy/api_client/init.py", line 438, in async_refresh_token raise TimeoutException() custom_components.octopus_energy.api_client.TimeoutException

Doing a curl command via terminal for any random website works (github.com for example), but trying to use the API calls to Octopus fails, even without any other parameters, e.g., curl https://api.octopus.energy times out. Doing this same command from any other machine on my local network works:

curl https://api.octopus.energy {"detail": "Not found."}%

Attempted numerous CURL commands to Octopus with and without parameters in the terminal and all fail with timeouts, but work outside the HA environment, yet other random sites work so very puzzled as to why it's just Octopus that fails within HA.

What version of Home Assistant Core has the issue?

core-2024.4.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

Octopus Energy (unable to re-install now)

Link to integration documentation on our website

No response

Diagnostics information

2024-04-17 08:30:25.629 WARNING (MainThread) [custom_components.octopus_energy.api_client] Failed to connect. Timeout of ClientTimeout(total=None, connect=None, sock_read=20, sock_connect=20, ceil_threshold=5) exceeded. 2024-04-17 08:30:25.630 ERROR (MainThread) [aiohttp.server] Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1025, in _wrap_create_connection return await self._loop.create_connection(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1102, in create_connection sock = await self._connect_sock( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1005, in _connect_sock await self.sock_connect(sock, address) File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 641, in sock_connect return await fut ^^^^^^^^^ asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 578, in _request conn = await self._connector.connect( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 544, in connect proto = await self._create_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 944, in _createconnection , proto = await self._create_direct_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1226, in _create_direct_connection transp, proto = await self._wrap_create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1022, in _wrap_create_connection async with ceil_timeout( File "/usr/local/lib/python3.12/asyncio/timeouts.py", line 115, in aexit raise TimeoutError from exc_val TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/octopus_energy/api_client/init.py", line 424, in async_refresh_token async with client.post(url, json=payload) as token_response: File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 1194, in aenter self._resp = await self._coro ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 582, in _request raise ServerTimeoutError( aiohttp.client_exceptions.ServerTimeoutError: Connection timeout to host https://api.octopus.energy/v1/graphql/

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request resp = await request_handler(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 543, in _handle resp = await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 114, in impl return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 83, in forwarded_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 88, in ban_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware response = await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle result = await handler(request, request.match_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 71, in with_admin return await func(self, request, *args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 285, in post return await super().post(request, flow_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 73, in wrapper result = await method(view, request, data, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 122, in post result = await self._flow_mgr.async_configure(flow_id, data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 356, in async_configure result = await self._async_configure(flow_id, user_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 402, in _async_configure result = await self._async_handle_step( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 506, in _async_handle_step result: _FlowResultT = await getattr(flow, method)(user_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/octopus_energy/config_flow.py", line 537, in async_step_user errors = await async_validate_main_config(config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/octopus_energy/config/main.py", line 72, in async_validate_main_config account_info = await client.async_get_account(data[CONFIG_ACCOUNT_ID]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/octopus_energy/api_client/init.py", line 442, in async_get_account await self.async_refresh_token() File "/config/custom_components/octopus_energy/api_client/init.py", line 438, in async_refresh_token raise TimeoutException() custom_components.octopus_energy.api_client.TimeoutException

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

Home Assistant running in Hyper-V VM with no hypervisor changes for 6+ months, everything worked fine until approx 15/4/2024

Version core-2024.4.3 Installation Type Home Assistant OS Development false Supervisor true Docker true User root Virtual Environment false Python Version 3.12.2 Operating System Family Linux Operating System Version 6.6.25-haos CPU Architecture x86_64 Timezone Europe/London Configuration Directory /config

Host Operating System Home Assistant OS 12.2 Update Channel stable Supervisor Version supervisor-2024.04.0 Agent Version 1.6.0 Docker Version 25.0.5 Disk Total 30.8 GB Disk Used 20.2 GB Healthy true Supported true Board ova Supervisor API ok Version API ok Installed Add-ons File editor (5.8.0), Terminal & SSH (9.13.0), Duck DNS (1.16.0), TasmoAdmin (0.30.0), Mosquitto broker (6.4.0)

SNCWilkinson commented 2 months ago

From HA Terminal: image

From a device on the same network: curl -v "https://api.octopus.energy"

mib1185 commented 2 months ago

Hi @SNCWilkinson This is a custom component, which is not maintained by the HA core project. Please report your issue to the maintainer or in the source repository of this custom component. Thx 👍

SNCWilkinson commented 2 months ago

This is an issue shown in a custom component, but the core is that the curl command is not working for this site. Raised it as it may be an issue others are experiencing and obviously I want it resolved myself.

Maybe it's an issue with just that specific site, however this only happens within HA, not on any other machine, so is it not a core HA issue?

mib1185 commented 2 months ago

https://github.com/home-assistant/core/issues/115748#issuecomment-2060875532 is not an issue of HA (there is no firewall or any other type of network filter running on HA) maybe your network configuration limits the outgoing connections of your HA instance 🤷‍♂️

SNCWilkinson commented 2 months ago

Today, with no changes, the curl command is working again and the integrations are now working.... Odd, and I'd love to know what happened, but it works now