search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.78k stars 30.04k forks source link

AEMET OpenData #107755

Closed kilem86 closed 7 months ago

kilem86 commented 8 months ago

The problem

Since I updated to the 2024.1 version of HA, the AEMET integration give me an error, It cannot be configured.

And I lost all weather details.

What version of Home Assistant Core has the issue?

2024.1.x

What was the last working version of Home Assistant Core?

2023.12.x

What type of installation are you running?

Home Assistant OS

Integration causing the issue

AEMET

Link to integration documentation on our website

No response

Diagnostics information

Logger: homeassistant.config_entries Source: config_entries.py:406 First occurred: 22:27:37 (1 occurrences) Last logged: 22:27:37

Error setting up entry AEMET for aemet Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 406, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/aemet/init.py", line 37, in async_setup_entry await aemet.select_coordinates(latitude, longitude) File "/usr/local/lib/python3.11/site-packages/aemet_opendata/interface.py", line 410, in select_coordinates town_data = await self.get_town_by_coordinates(latitude, longitude) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/aemet_opendata/interface.py", line 370, in get_town_by_coordinates towns = await self.get_towns() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/aemet_opendata/interface.py", line 390, in get_towns return await self.api_call("maestro/municipios") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/aemet_opendata/interface.py", line 135, in api_call resp: ClientResponse = await self.aiohttp_session.request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 500, in _request with timer: File "/usr/local/lib/python3.11/site-packages/aiohttp/helpers.py", line 735, in exit raise asyncio.TimeoutError from None TimeoutError

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 8 months ago

Hey there @noltari, mind taking a look at this issue as it has been labeled with an integration (aemet) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `aemet` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign aemet` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)

aemet documentation aemet source (message by IssueLinks)

Noltari commented 8 months ago

Maybe the AEMET OpenData API was down at that time. @kilem86 can you check again?

kilem86 commented 8 months ago

Yes, I just checked and it is the same, sometimes the data are available, and sometimes not.

The problem is that I have some sensors related to the data, and when is not available, I lost the sensor´s information.

kilem86 commented 8 months ago

image

Noltari commented 8 months ago

Yes, I just checked and it is the same, sometimes the data are available, and sometimes not.

The problem is that I have some sensors related to the data, and when is not available, I lost the sensor´s information.

I know, but this is the expected behaviour. If the AEMET OpenData API is unstable then the problem isn't related to the integration.

afharo commented 7 months ago

I was on my way to report a similar issue, but I thought I'd post here because I think it's related. Please, let me know if you think I should open it separately.

My logs:

Logger: homeassistant.config_entries
Source: config_entries.py:444
First occurred: 11 February 2024 at 23:45:46 (1 occurrences)
Last logged: 11 February 2024 at 23:45:46

Error setting up entry AEMET for aemet
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 444, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/aemet/__init__.py", line 37, in async_setup_entry
    await aemet.select_coordinates(latitude, longitude)
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 399, in select_coordinates
    await self.get_conventional_observation_station_by_coordinates(
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 305, in get_conventional_observation_station_by_coordinates
    stations = await self.get_conventional_observation_stations()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 298, in get_conventional_observation_stations
    return await self.api_call("observacion/convencional/todas", fetch_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 135, in api_call
    resp: ClientResponse = await self.aiohttp_session.request(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 504, in _request
    with timer:
  File "/usr/local/lib/python3.12/site-packages/aiohttp/helpers.py", line 735, in __exit__
    raise asyncio.TimeoutError from None
TimeoutError

It's been happening to me for a few releases: every time I restart HASS the integration fails to setup due to timeouts. If I wait for HASS to settle and then reload this integration, it successfully loads.

FWIW, I can see many other timeout errors coming from other integrations (prob my Raspberry Pi is not able to handle all the startup requests and they just timeout).

I wonder if this is a matter of retries: I have the feeling (I haven't checked the code) that, if the integration fails to setup, it stays as failed forever and never retries.

I worked around this with an automation that runs 5 minutes after HASS started, checks if AEMET's state is unavailable and reloads the integration. Then, it waits 1 minute, and checks again. It repeats this up to 60 times before giving up. Usually, on the first attempt, it goes through.

TL;DR, is it worth adding a retry logic on timeouts to the integration's setup process?

Thank you @Noltari for your integration 😇

figorr commented 7 months ago

I was on my way to report a similar issue, but I thought I'd post here because I think it's related. Please, let me know if you think I should open it separately.

My logs:

Logger: homeassistant.config_entries
Source: config_entries.py:444
First occurred: 11 February 2024 at 23:45:46 (1 occurrences)
Last logged: 11 February 2024 at 23:45:46

Error setting up entry AEMET for aemet
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 444, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/aemet/__init__.py", line 37, in async_setup_entry
    await aemet.select_coordinates(latitude, longitude)
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 399, in select_coordinates
    await self.get_conventional_observation_station_by_coordinates(
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 305, in get_conventional_observation_station_by_coordinates
    stations = await self.get_conventional_observation_stations()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 298, in get_conventional_observation_stations
    return await self.api_call("observacion/convencional/todas", fetch_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 135, in api_call
    resp: ClientResponse = await self.aiohttp_session.request(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 504, in _request
    with timer:
  File "/usr/local/lib/python3.12/site-packages/aiohttp/helpers.py", line 735, in __exit__
    raise asyncio.TimeoutError from None
TimeoutError

It's been happening to me for a few releases: every time I restart HASS the integration fails to setup due to timeouts. If I wait for HASS to settle and then reload this integration, it successfully loads.

FWIW, I can see many other timeout errors coming from other integrations (prob my Raspberry Pi is not able to handle all the startup requests and they just timeout).

I wonder if this is a matter of retries: I have the feeling (I haven't checked the code) that, if the integration fails to setup, it stays as failed forever and never retries.

I worked around this with an automation that runs 5 minutes after HASS started, checks if AEMET's state is unavailable and reloads the integration. Then, it waits 1 minute, and checks again. It repeats this up to 60 times before giving up. Usually, on the first attempt, it goes through.

TL;DR, is it worth adding a retry logic on timeouts to the integration's setup process?

Thank you @Noltari for your integration 😇

Similar Issue since I updated Home Assistant to 2024.2.

Every time I restarted HA ... the integration fails to load ... so I need to reload. Usually one reload is enough ... but sometimes it fails several times and then stays as failed. So it needs a new reload after a while.

Another strange behaviour is ... every day at 19:00h seems to be an error recovering some data (prepitation probability, storm probability and snow probability) ... and these sensors become "unavailable" or "unknown". And they became unavailable until midnight even if I reload the integration. I though this could be a problem with the station that provides the data ... but the weird thing is they stay as "unavailable" or "unknown" forever unless I reload the integration after the midnight.

Before the midnight ... the reload didn't recover these sensors. After midnight if I reload the integration these sensors are recovered and they started to show the correct data again.

Noltari commented 7 months ago

Another strange behaviour is ... every day at 19:00h seems to be an error recovering some data (prepitation probability, storm probability and snow probability) ... and these sensors become "unavailable" or "unknown". And they became unavailable until midnight even if I reload the integration. I though this could be a problem with the station that provides the data ... but the weird thing is they stay as "unavailable" or "unknown" forever unless I reload the integration after the midnight.

Before the midnight ... the reload didn't recover these sensors. After midnight if I reload the integration these sensors are recovered and they started to show the correct data again.

@figorr this is a different issue, so please create a new one instead of reporting it on an unrelated issue.

figorr commented 7 months ago

Ok, thanks @Noltari. I will open the new issue this afternoon to see if I can see anything in the logs.

Noltari commented 7 months ago

Same problem

@rdlvm no info at all provided, so please refrain from doing this in the future. Doing this just bothers the person that's going to fix it, so the only thing you're getting with this kind of comments is a delay.

afharo commented 7 months ago

I worked around this with an automation that runs 5 minutes after HASS started, checks if AEMET's state is unavailable and reloads the integration. Then, it waits 1 minute, and checks again. It repeats this up to 60 times before giving up. Usually, on the first attempt, it goes through.

In case anyone is interested in the workaround, this is my definition in yaml:

alias: "[HASS] Ensure AEMET works"
description: ""
trigger:
  - platform: homeassistant
    event: start
condition: []
action:
  - delay:
      hours: 0
      minutes: 2
      seconds: 0
      milliseconds: 0
  - alias: Ensure AEMET works
    repeat:
      sequence:
        - service: homeassistant.reload_config_entry
          target:
            entity_id: weather.aemet
          data: {}
          continue_on_error: true
        - delay:
            hours: 0
            minutes: 1
            seconds: 0
            milliseconds: 0
      while:
        - condition: and
          conditions:
            - condition: state
              entity_id: weather.aemet
              state: unavailable
            - condition: template
              value_template: "{{ repeat.index < 60 }}"
              alias: Less than 60 iterations
    enabled: true
mode: single
diamant-x commented 7 months ago

I'm having the same issue as Original post. This is the logs when enabling debugging in the integration:

2024-02-21 10:52:48.922 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry AEMET for aemet
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 444, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/aemet/__init__.py", line 37, in async_setup_entry
    await aemet.select_coordinates(latitude, longitude)
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 399, in select_coordinates
    await self.get_conventional_observation_station_by_coordinates(
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 305, in get_conventional_observation_station_by_coordinates
    stations = await self.get_conventional_observation_stations()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 298, in get_conventional_observation_stations
    return await self.api_call("observacion/convencional/todas", fetch_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 135, in api_call
    resp: ClientResponse = await self.aiohttp_session.request(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 504, in _request
    with timer:
  File "/usr/local/lib/python3.12/site-packages/aiohttp/helpers.py", line 735, in __exit__
    raise asyncio.TimeoutError from None
TimeoutError

If i reload the integration it connects and retrieves it, but not permanently; after some time it fails again. I even tried deleting it, restarting, requesting a new API key in aemet and reconfiguring it from scratch, but that won't help either. At the same time I started seeing another issue which might point towards Python 3.12 not being supported maybe? https://github.com/home-assistant/core/issues/110538#issuecomment-1952954770

Below my system data. This issue happens even if I restart in Safe Mode image

Thanks.

System Information

version core-2024.2.2
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.12.1
os_name Linux
os_version 6.1.63-haos-raspi
arch aarch64
timezone Europe/Madrid
config_dir /config
Home Assistant Cloud logged_in | false -- | -- can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok
Home Assistant Supervisor host_os | Home Assistant OS 11.5 -- | -- update_channel | stable supervisor_version | supervisor-2024.02.0 agent_version | 1.6.0 docker_version | 24.0.7 disk_total | 116.5 GB disk_used | 72.5 GB healthy | true supported | true board | rpi4-64 supervisor_api | ok version_api | ok installed_addons | AdGuard Home (5.0.3), ESPHome (2024.2.0), FTP (5.0.1), Firefly iii (6.1.9), Firefly iii Data Importer (1.4.3), Home Assistant Google Drive Backup (0.112.1), InfluxDB2 (0.1.0.0), WireGuard (0.10.2), Nginx Proxy Manager (1.0.1), Samba share (12.3.0), Prowlarr NAS (nightly-1.14.0.4286-ls116), Vaultwarden (Bitwarden) (0.21.1), chrony (3.0.1), motionEye (0.20.0), qBittorrent (4.6.3-4), Sonarr (4.0.1.1131), Radarr (5.2.6.8376), Lidarr NAS (2.1.7.4030), Plex Media Server (3.5.0), MariaDB (2.6.1), Bazarr NAS (1.4.1), DOODS2 (1), Castsponsorskip (0.7.2), phpMyAdmin (0.9.1), File editor (5.8.0), Traccar (0.25.0)
Dashboards dashboards | 7 -- | -- resources | 17 views | 17 mode | storage
Recorder oldest_recorder_run | 23 de diciembre de 2023, 09:42 -- | -- current_recorder_run | 21 de febrero de 2024, 10:52 estimated_db_size | 4023.14 MiB database_engine | mysql database_version | 10.6.12