search

caiosweet / Home-Assistant-custom-components-INGV

INGV - National Institute of Geophysics and Volcanology [Istituto Nazionale di Geofisica e Vulcanologia] Terremoti Italia.
http://www.ingv.it/it/
MIT License
23 stars 7 forks source link

Il componente non rimuove le entità precedenti #20

Closed Cionco65 closed 2 years ago

Cionco65 commented 2 years ago

Chiedo scusa per l'uso dell'italiano ma non ho troppa dimestichezza con l'inglese. Ho correttamente installato tramite HACS il componente, le entità vengono generate ma non vengono mai rimosse. Al momento ne ho 4 e se dovessero arrivare altri eventi credo aumenteranno.

Il mio sistema è HassOS Supervised, Core 2022.2.9, Python 3.9.7 su Raspberry 4

Non saprei come riprodurre l'errore e non ho visto nulla nei log che possa aiutarmi nella comprensione.

Grazie per l'ottimo lavoro Paolo

caiosweet commented 2 years ago

Ciao, va benissimo l'italiano :0) Le entità, vengono rimosse quando non sono più nella lista del feed, in base al filtro magnitudo. Il feed di base è questo riferito alla lista terremoti

Premetto che non sono un programmatore e ho realizzato questo componente sulla base di quello Spagnolo ign_sismologia

Cionco65 commented 2 years ago

Ah ok, non avevo capito, pensavo venissero rimosse ad ogni evento nuovo. Io mi arrabatto un po' nella programmazione ed in effetti sono andato a vedere il codice, che mi ha dato anche degli ottimi spunti.

Grazie per la celerità e la disponibilità. Paolo

caiosweet commented 2 years ago

Figurati! Anzi, se trovi il modo di aggiungere il campo depth, sarebbe una buona aggiunta :D Essendomi basato sul lavoro di @exxamalte che utilizza quel feed, non c'è modo di avere quel campo, se non richiamando l'evento con il codice id (quindi altra richiesta web)

Però, ho notato che c'è anche un'altra strada (almeno credo) per ricavare i dati, ed è openapi con conseguente rate limit da gestire... :)

exxamalte commented 2 years ago

Purtroppo il GeoRSS feed che uso non contiene il profondità. Ma, questo openapi ritorna QuakeML e include profondità. Credo che il rate limit è 10 richieste per secondo per IP? Sarebbe possibile di sviluppare un integrazione, simile alla corrente.

caiosweet commented 2 years ago

Thanks for your interest. Yes, it would be more complete to have depth too, but I'm just not able to help you :( Finally, I don't know, which resource is better to use as a feed (the one you currently use or this one with QuakeM)

Anyway... I honestly don't know the time limit for rate-limit. The information I found here ingv webservices and software and project on github

From what I see, the header, it looks like yes, 10 requests per second?

HTTP/1.1 200 OK
Server: nginx
Date: Sun, 27 Feb 2022 13:06:26 GMT
Content-Type: application/xml
Transfer-Encoding: chunked
Connection: keep-alive
Keep-Alive: timeout=65
Vary: Accept-Encoding
Access-Control-Allow-Origin: *
Content-Encoding: gzip
X-RateLimit-Limit: 10
X-RateLimit-Reset: 1
X-UA-Compatible: IE=Edge
X-Cache-Status: HIT
Cache-Control: public, max-age=60
Cionco65 commented 2 years ago

Credo di non essere ad un livello sufficiente di preparazione per poter affrontare tali argomenti

caiosweet commented 2 years ago

Tranquillo, l'importante è che hai trovato la risposta che cercavi. Chiudo.

exxamalte commented 2 years ago

Nel frattempo ho sviluppato un nuovo integrazione usare il INGV QuakeML feed: https://github.com/exxamalte/python-aio-quakeml-ingv-centro-nazionale-terremoti-client Purtroppo non è possibile di solo sostituire il vecchio Python library con il nuovo, ma posso darti aiuto se vuoi.

caiosweet commented 2 years ago

I copied the old library from ign_sismologia :D I can try, I have to study it, maybe I look at some of your work that you have already done :0)

caiosweet commented 2 years ago

Nel frattempo ho sviluppato un nuovo integrazione usare il INGV QuakeML feed: https://github.com/exxamalte/python-aio-quakeml-ingv-centro-nazionale-terremoti-client Purtroppo non è possibile di solo sostituire il vecchio Python library con il nuovo, ma posso darti aiuto se vuoi.

I wanted to thank you for the work you have done. Based on GeoNet NZ Quakes, I started producing some results: 0)

Sensor Schermata 2022-05-16 alle 23 44 05

Geo location Schermata 2022-05-16 alle 23 44 58

exxamalte commented 2 years ago

That's great news. The GeoNet NZ Quakes integration is actually a pretty good one to use as a blueprint for this. Let me know if there is anything missing in the new QuakeML based library.

caiosweet commented 2 years ago

Hello. Before asking you a couple of tips, I found some problems with the geolocation trigger for each geo_location component (Gdacs, GeoNet .. etc.)

In short:

Do you think I should open an issue?

In the meantime, I have used your components as a blueprint and I am undecided for which version will I have to use (manager or coordinator) ... in any case, I have made progress :0)

##### GEONET START DEBUG

2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Feed entity manager added for 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Feed entity manager initialized
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Sensor setup done
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Waiting for updates 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Updating 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Geolocation setup done
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Status update received: <StatusUpdate(OK@2022-05-22 18:04:27.714272)>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Feed entity manager updated
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Received status update for 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Updating 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p374660
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p369488
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p379226
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p376125
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p366188
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p380263
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p366254
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p380786
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p380295
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p364037
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p381965
2022-05-22 18:04:27 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p368516
2022-05-22 18:04:27 INFO (MainThread) [homeassistant.bootstrap] Home Assistant initialized in 1.09s

#reload from UI

2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Feed entity manager stopped
2022-05-22 18:04:41 ERROR (MainThread) [homeassistant.helpers.event] Error while processing state change for geo_location.20_km_north_west_of_taihape
Traceback (most recent call last):
  File "/Users/caio/Documents/hass/core/venv/lib/python3.10/site-packages/homeassistant/helpers/event.py", line 271, in _async_state_change_dispatcher
    hass.async_run_hass_job(job, event)
  File "/Users/caio/Documents/hass/core/venv/lib/python3.10/site-packages/homeassistant/core.py", line 523, in async_run_hass_job
    cast(Callable[..., _R], hassjob.target)(*args)
  File "/Users/caio/Documents/hass/core/venv/lib/python3.10/site-packages/homeassistant/components/geo_location/trigger.py", line 67, in state_change_listener
    to_match = condition.zone(hass, zone_state, to_state) if to_state else False
  File "/Users/caio/Documents/hass/core/venv/lib/python3.10/site-packages/homeassistant/helpers/condition.py", line 836, in zone
    raise ConditionErrorMessage(
homeassistant.exceptions.ConditionErrorMessage: In 'zone' condition: entity geo_location.20_km_north_west_of_taihape has no 'latitude' attribute

.... cut ....

2022-05-22 18:04:41 ERROR (MainThread) [homeassistant.helpers.event] Error while processing state change for geo_location.5_km_north_west_of_porangahau
Traceback (most recent call last):
  File "/Users/caio/Documents/hass/core/venv/lib/python3.10/site-packages/homeassistant/helpers/event.py", line 271, in _async_state_change_dispatcher
    hass.async_run_hass_job(job, event)
  File "/Users/caio/Documents/hass/core/venv/lib/python3.10/site-packages/homeassistant/core.py", line 523, in async_run_hass_job
    cast(Callable[..., _R], hassjob.target)(*args)
  File "/Users/caio/Documents/hass/core/venv/lib/python3.10/site-packages/homeassistant/components/geo_location/trigger.py", line 67, in state_change_listener
    to_match = condition.zone(hass, zone_state, to_state) if to_state else False
  File "/Users/caio/Documents/hass/core/venv/lib/python3.10/site-packages/homeassistant/helpers/condition.py", line 836, in zone
    raise ConditionErrorMessage(
homeassistant.exceptions.ConditionErrorMessage: In 'zone' condition: entity geo_location.5_km_north_west_of_porangahau has no 'latitude' attribute
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Feed entity manager added for 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Feed entity manager initialized
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Sensor setup done
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Geolocation setup done
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Waiting for updates 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Updating 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Status update received: <StatusUpdate(OK@2022-05-22 18:04:41.091368)>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes] Feed entity manager updated
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Received status update for 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.sensor] Updating 4a2b91b5c07016c55cfb6033acf5403a
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p374660
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p369488
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p379226
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p376125
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p366188
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p380263
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p366254
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p380786
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p380295
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p364037
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p381965
2022-05-22 18:04:41 DEBUG (MainThread) [homeassistant.components.geonetnz_quakes.geo_location] Updating 2022p368516
# ERROR Connection
2022-05-18 21:55:12 DEBUG (MainThread) [custom_components.ingv_quakes] Status update received: <StatusUpdate(OK@2022-05-18 21:55:12.825413)>
2022-05-18 21:55:12 DEBUG (MainThread) [custom_components.ingv_quakes] Feed entity manager updated
2022-05-18 21:55:12 DEBUG (MainThread) [custom_components.ingv_quakes.sensor] Received status update for 5c0dde1cd02bc240d5003e625267beac
2022-05-18 21:55:12 DEBUG (MainThread) [custom_components.ingv_quakes.geo_location] Updating smi:webservices.ingv.it/fdsnws/event/1/query?eventId=31109961
2022-05-18 21:55:12 DEBUG (MainThread) [custom_components.ingv_quakes.sensor] Updating 5c0dde1cd02bc240d5003e625267beac
2022-05-18 22:00:23 WARNING (MainThread) [aio_quakeml_client.feed] Requesting data from https://webservices.ingv.it/fdsnws/event/1/query?lat=REDACT&lon=REDACT&maxradiuskm=1000&minmag=3.0&starttime=2022-05-17T20:00:00 failed with timeout error
2022-05-18 22:00:23 WARNING (MainThread) [aio_quakeml_client.feed_manager] Update not successful, no data received from <IngvCentroNazionaleTerremotiQuakeMLFeed(home=(REDACT, REDACT), url=https://webservices.ingv.it/fdsnws/event/1/query, radius=1000, magnitude=3.0)>
2022-05-18 22:00:23 DEBUG (MainThread) [custom_components.ingv_quakes] Status update received: <StatusUpdate(ERROR@2022-05-18 22:00:23.011033)>
2022-05-18 22:00:23 DEBUG (MainThread) [custom_components.ingv_quakes] Feed entity manager updated
2022-05-18 22:00:23 DEBUG (MainThread) [custom_components.ingv_quakes.sensor] Received status update for 5c0dde1cd02bc240d5003e625267beac
2022-05-18 22:00:23 DEBUG (MainThread) [custom_components.ingv_quakes.sensor] Updating 5c0dde1cd02bc240d5003e625267beac
2022-05-18 22:05:12 DEBUG (MainThread) [custom_components.ingv_quakes] Status update received: <StatusUpdate(OK@2022-05-18 22:05:12.853402)>
2022-05-18 22:05:12 DEBUG (MainThread) [custom_components.ingv_quakes] Feed entity manager updated
2022-05-18 22:05:12 DEBUG (MainThread) [custom_components.ingv_quakes.geo_location] Adding geolocation <Entity None: None>
2022-05-18 22:05:12 DEBUG (MainThread) [custom_components.ingv_quakes.sensor] Received status update for 5c0dde1cd02bc240d5003e625267beac
2022-05-18 22:05:12 DEBUG (MainThread) [custom_components.ingv_quakes.sensor] Updating 5c0dde1cd02bc240d5003e625267beac
2022-05-18 22:05:12 DEBUG (MainThread) [custom_components.ingv_quakes.geo_location] Updating smi:webservices.ingv.it/fdsnws/event/1/query?eventId=31109961

# Simple automation from UI

alias: New Geonet Quake
description: ''
trigger:
  - platform: geo_location
    source: geonetnz_quakes
    zone: zone.geoalert
    event: enter
condition:
  - condition: template
    value_template: '{{ trigger.to_state.attributes.latitude is defined }}'
action:
  - service: persistent_notification.create
    data:
      notification_id: geonet
      title: New Quake
      message: Test
mode: queued
max: 20
exxamalte commented 2 years ago
  • gives an error when reloading the component from the UI (see the log below) with a simple automation. (with both enter or leave events). I also tried to use conditions, but nothing.

Yes, I can reprodue this issue. Looks like a bit of a race condition where the automation is triggered while the geolocation entities are being removed and then added. I just opened: https://github.com/home-assistant/core/issues/72356

  • repeats the trigger if there have been disconnections between a failed update and a successful one. But this I think is due to the component, which removes and adds the same entities, without having a trace.

Yes, that is how these integrations work unfortunately. I am currently not distinguishing the cases where a single entity is missing from a successful feed update vs. the whole feed becomes unavailable.

In the meantime, I have used your components as a blueprint and I am undecided for which version will I have to use (manager or coordinator)

I have started migrating integrations to use the coordinator pattern because that's a standard way to update entities.

caiosweet commented 2 years ago

ma posso darti aiuto se vuoi.

Hi. Before asking for your help, I tried to do something by myself: D First of all thank you for solving the geo location trigger problem: 0)

After 3 weeks I noticed something strange that I can not solve. The last event, is not removed if the delta of the configurable start time has elapsed (I tried with 24 hours and even less with 6 hours)

Everything works fine, when there are two or more events, but when only the last one remains it does not clear from the registry of devices. But if I do a reload of the component, it deletes, as it should be.

When you have time and desire, could you help me or show me where I can look?

Let me know if there is anything missing in the new QuakeML based library.

I made the component according to your project geo_json_events, something changed, but not so much.

I simply "preferred" not to use the coordinates as a unique ID for the entity_id sensor and got some attributes not quite right (to avoid bothering you again) but it works. I used a shorter name (geo_location.ingv_earthquakes_EVENTID) for geolocation entities. (in the future I would like to change the domain, use a shorter one e.g. INGV Earthquakes or INGV Quakes or leave everything as I have already done :D)

Anyway, thank you for your concern. 🇦🇺 👍

exxamalte commented 2 years ago

I simply "preferred" not to use the coordinates as a unique ID for the entity_id sensor and got some attributes not quite right (to avoid bothering you again) but it works.

That's OK in general. The main reason I usually use the coordinates is because that literally helps making the unique ID unique.

I used a shorter name (geo_location.ingv_earthquakes_EVENTID) for geolocation entities.

That might be a good decision. I believe there is a maximum character length for entity ids, so keeping them as short and ideally unique as possible helps.

Everything works fine, when there are two or more events, but when only the last one remains it does not clear from the registry of devices. But if I do a reload of the component, it deletes, as it should be.

Yeah, I think I've seen behaviour like that before, but only in my development environment. The nature of the asnychronous setup comes with the disadvantage that not all deletions happen instantly. However, from my experience these orphaned entities normally are nothing to worry about, and will disappear eventually.