Why is powercalc trying to reach the internet after installation and setup?

fullerms commented 3 days ago

System Health details

System Information

version	core-2024.7.0
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.12.4
os_name	Linux
os_version	6.6.31-haos-raspi
arch	aarch64
timezone	Australia/Sydney
config_dir	/config

Home Assistant Community Store

Home Assistant Cloud

Home Assistant Supervisor

Dashboards

Recorder

Sonoff

version | 3.7.3 (e240aaf) -- | -- cloud_online | 0 / 1 local_online | 0 / 0

Checklist

[X] I have read the FAQ to see if there is a known solution to my problem.
[X] I have enabled debug logging for my installation.
[X] I have filled out the issue template to the best of my ability.
[X] This issue only contains 1 issue (if you have multiple issues, open one issue for each issue).
[X] This issue is not a duplicate issue of currently previous issues..

Describe the issue

Had an internet outage for a couple of hours today and noticed that PowerCalc sensors were not available anymore, Checked the logs and found these entries.

Failed to download, retrying... (Attempt 1 of 3) Failed to download, retrying... (Attempt 2 of 3)

Reproduction steps

1.Disconnect internet from main router 2. 3. ...

Debug logs

NA

Diagnostics dump or YAML config

No response

bramstroker commented 3 days ago

@fullerms Powercalc complete profile library previously was included in the component, but it's size was growing out of control. So since a few months powercalc downloads the needed profiles from the internet. It should only do that once and load locally when it has already downloaded. On every startup powercalc downloads meta data about the full library to see if certain profiles have been updated and than it should redownload.

It seems you got an error on downloading this metadata, but this should also fallback to local to not break the powercalc setup.

I will have a look into this to see why this fallback was not working for you and to make it more robust.

Also more information about the library and download feature is here: https://homeassistant-powercalc.readthedocs.io/en/latest/library/library.html

tvwerkhoven commented 3 days ago

I have a related issue: my powercalc v1.12.1 fails to start when there is no internet when starting Home Assistant. It tried to connect to the internet, fails, and then the sensors are unavailable.

This is the log file:

``` 2024-06-08 13:58:04.363 ERROR (MainThread) [homeassistant.setup] Error during setup of component powercalc Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1203, in _create_direct_connection hosts = await self._resolve_host(host, port, traces=traces) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 880, in _resolve_host return await asyncio.shield(resolved_host_task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 917, in _resolve_host_with_throttle addrs = await self._resolver.resolve(host, port, family=self._family) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/resolver.py", line 33, in resolve infos = await self._loop.getaddrinfo( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 899, in getaddrinfo return await self.run_in_executor( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/socket.py", line 963, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Try again The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/setup.py", line 404, in _async_setup_component result = await task ^^^^^^^^^^ File "/config/custom_components/powercalc/__init__.py", line 238, in async_setup await discovery_manager.start_discovery() File "/config/custom_components/powercalc/discovery.py", line 123, in start_discovery power_profile = await self.get_power_profile( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/powercalc/discovery.py", line 144, in get_power_profile self.power_profiles[entity_id] = await get_power_profile( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/powercalc/power_profile/factory.py", line 39, in get_power_profile library = await ProfileLibrary.factory(hass) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/powercalc/power_profile/library.py", line 49, in factory await library.initialize() File "/config/custom_components/powercalc/power_profile/library.py", line 35, in initialize await self._loader.initialize() File "/config/custom_components/powercalc/power_profile/loader/composite.py", line 14, in initialize [await loader.initialize() for loader in self.loaders] # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/powercalc/power_profile/loader/remote.py", line 33, in initialize self.library_contents = await self.load_library_json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/powercalc/power_profile/loader/remote.py", line 53, in load_library_json async with aiohttp.ClientSession() as session, session.get(ENDPOINT_LIBRARY) as resp: File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 1197, in __aenter__ self._resp = await self._coro ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 581, in _request conn = await self._connector.connect( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 544, in connect proto = await self._create_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 944, in _create_connection _, proto = await self._create_direct_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1209, in _create_direct_connection raise ClientConnectorError(req.connection_key, exc) from exc aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host powercalc.lauwbier.nl:443 ssl:default [Try again] ```

I like the option to manually download the libraries. Perhaps it's an idea to download once, and make the check for updates opt-in/opt-out/manual/(more) robust against lack of internet.

P.S. really like your add-on :)

bramstroker commented 2 days ago

There were some changes to make things more robust in v1.12.x revisions, and better retry mechanism and error handling. I improved more on it and just released v1.13.1 which contains latest improvement in logging. The error you got @tvwerkhoven were fixed in recent versions of powercalc.

I tried myself by disconnecting the internet and then it should just fallback to local files which were already downloaded before, and successfully startup powercalc. So in essence it should work like you want @tvwerkhoven with the latest version.

When you never want Powercalc to attempt connecting to the internet ever you can use this configuration: https://homeassistant-powercalc.readthedocs.io/en/latest/library/library.html#disable-download-feature But that also means you'll need to manage all the downloading and copying of the profiles manually.

Could you please try again with v1.13.1 and let me know if this starts up correctly without internet access? When still problems please also enable debug logging and share the logs after startup.

fullerms commented 2 days ago

@bramstroker thanks for the prompt response. Will test after upgrading to the new release and let you know.

bramstroker / homeassistant-powercalc