search

BJReplay / ha-solcast-solar

Solcast Integration for Home Assistant
Apache License 2.0
146 stars 31 forks source link

Probable issues initialising cache due to likely persistent 429s from Solcast Service - Logging required. #32

Closed pajikcz closed 2 months ago

pajikcz commented 2 months ago

Describe the bug just migrated from ozzie to BJreplay integration and cannot fetch data, API key added but is taking too long to initializace and no data fetched, no restart helped.

data are OK on solcast https://toolkit.solcast.com.au/home-pv-system/

message from integration: Solcast PV Forecast API Used became unavailable

running HAOS in all latest versions

Core 2024.6.3 Supervisor 2024.06.0 Operating System 12.3 Frontend 20240610.1

BJReplay commented 2 months ago

OK, I can repro.


docker run -d   --name=homeassistantls  -e PUID=1000   -e PGID=1000   -e TZ=Australia/Melbourne -p 8124:8123  -v ./hals:/config lscr.io/linuxserver/homeassistant:latest
docker exec -it homeassistantls /bin/bash
ls
root@2fd2f1d6ad2a:/# ls
app  bin  command  config  defaults  dev  docker-mods  etc  home  init  lib  lsiopy  media  mnt  opt  package  proc  root  run  sbin  srv  sys  tmp  usr  var
root@2fd2f1d6ad2a:/# cd config
root@2fd2f1d6ad2a:/config# ls
automations.yaml  blueprints  configuration.yaml  deps  home-assistant.log  home-assistant.log.1  home-assistant.log.fault  home-assistant_v2.db  home-assistant_v2.db-shm  home-assistant_v2.db-wal  scenes.yaml  scripts.yaml  secrets.yaml  tts
root@2fd2f1d6ad2a:/config# mkdir custom_components
root@2fd2f1d6ad2a:/config# cd~ 
root@2fd2f1d6ad2a:/# git clone https://github.com/BJReplay/ha-solcast-solar
Cloning into 'ha-solcast-solar'...
remote: Enumerating objects: 2207, done.
remote: Counting objects: 100% (588/588), done.
remote: Compressing objects: 100% (166/166), done.
remote: Total 2207 (delta 469), reused 480 (delta 398), pack-reused 1619
Receiving objects: 100% (2207/2207), 926.06 KiB | 3.09 MiB/s, done.
Resolving deltas: 100% (1378/1378), done.
root@2fd2f1d6ad2a:/# cd ha-solcast-solar/
root@2fd2f1d6ad2a:/ha-solcast-solar# ls
CODEOWNERS  LICENSE.md  README.md  custom_components  hacs.json
root@2fd2f1d6ad2a:/ha-solcast-solar#  cd custom_components/
root@2fd2f1d6ad2a:/ha-solcast-solar/custom_components# ls -al
total 12
drwxr-xr-x 3 root root 4096 Jun 18 16:09 .
drwxr-xr-x 5 root root 4096 Jun 18 16:09 ..
drwxr-xr-x 3 root root 4096 Jun 18 16:09 solcast_solar
root@2fd2f1d6ad2a:/ha-solcast-solar/custom_components# mv solcast_solar/ /config/custom_components/

Another thought, is solcast-usage.json getting created in /config?

No.


root@2fd2f1d6ad2a:/config# ls -al
total 772
drwxr-xr-x 8 abc  users   4096 Jun 18 16:11 .
drwxr-xr-x 1 root root    4096 Jun 18 16:09 ..
-rw-r--r-- 1 abc  users      8 Jun 18 15:51 .HA_VERSION
drwxr-xr-x 2 abc  users   4096 Jun 18 15:51 .cloud
drwxr-xr-x 2 abc  users   4096 Jun 18 16:14 .storage
-rw-r--r-- 1 abc  users      2 Jun 18 15:51 automations.yaml
drwxr-xr-x 4 abc  users   4096 Jun 18 15:51 blueprints
-rw-r--r-- 1 abc  users    265 Jun 18 15:51 configuration.yaml
drwxr-xr-x 3 root root    4096 Jun 18 16:10 custom_components
drwxr-xr-x 2 abc  users   4096 Jun 18 15:51 deps
-rw-r--r-- 1 abc  users   4995 Jun 18 16:12 home-assistant.log
-rw-r--r-- 1 abc  users      0 Jun 18 15:58 home-assistant.log.1
-rw-r--r-- 1 abc  users      0 Jun 18 16:11 home-assistant.log.fault
-rw-r--r-- 1 abc  users 253952 Jun 18 16:11 home-assistant_v2.db
-rw-r--r-- 1 abc  users  32768 Jun 18 16:15 home-assistant_v2.db-shm
-rw-r--r-- 1 abc  users 444992 Jun 18 16:15 home-assistant_v2.db-wal
-rw-r--r-- 1 abc  users      0 Jun 18 15:51 scenes.yaml
-rw-r--r-- 1 abc  users      0 Jun 18 15:51 scripts.yaml
-rw-r--r-- 1 abc  users    161 Jun 18 15:51 secrets.yaml
drwxr-xr-x 2 abc  users   4096 Jun 18 15:51 tts

root@2fd2f1d6ad2a:/config# cat home-assistant.log
2024-06-18 16:11:25.670 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration solcast_solar which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-06-18 16:11:59.113 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] SOLCAST - sites_data Exception error: Traceback (most recent call last):
  File "/config/custom_components/solcast_solar/solcastapi.py", line 148, in sites_data
    async with aiofiles.open(apiCacheFileName, 'w') as f:
  File "/usr/local/lib/python3.12/site-packages/aiofiles/base.py", line 98, in __aenter__
    self._obj = await self._coro
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiofiles/threadpool/__init__.py", line 94, in _open
    f = yield from loop.run_in_executor(executor, cb)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/util/loop.py", line 131, in protected_loop_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'solcast-sites.json'

2024-06-18 16:11:59.154 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] SOLCAST - sites_usage error: Traceback (most recent call last):
  File "/config/custom_components/solcast_solar/solcastapi.py", line 215, in sites_usage
    async with aiofiles.open(apiCacheFileName, 'w') as f:
  File "/usr/local/lib/python3.12/site-packages/aiofiles/base.py", line 98, in __aenter__
    self._obj = await self._coro
                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiofiles/threadpool/__init__.py", line 94, in _open
    f = yield from loop.run_in_executor(executor, cb)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/util/loop.py", line 131, in protected_loop_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'solcast-usage.json'

2024-06-18 16:12:28.103 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 543, in _handle
    resp = await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 114, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/components/http/forwarded.py", line 83, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/components/http/ban.py", line 85, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/components/http/auth.py", line 242, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/homeassistant/components/diagnostics/__init__.py", line 280, in get
    data = await info.config_entry_diagnostics(hass, config_entry)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/diagnostics.py", line 32, in async_get_config_entry_diagnostics
    "energy_forecasts_graph": coordinator.solcast._dataenergy["wh_hours"],
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'wh_hours'

What do we know?

A brand new pull and set up of linuxserver/homeassistant and a brand new manual install of solcast repros.

It looks like there is a permissions problems - possibly with aiofiles.open for write. I note that linuxserver notes that: Rebase to alpine 3.19, upgrade to python 3.12. Restructure python packages so all packages are internal (no more venv in /config). Use upstream project's wheels. Due to an upstream issue, on first start of the newly updated container, some custom integrations may be disabled in HA due to missing dependencies. A subsequent container restart should fix that and the integrations should be re-enabled.

BJReplay commented 2 months ago

Closing - opening new issue - this is getting too far away from original issue.