jabesq-org / pyatmo

Simple API to access Netatmo weather station data from any python script (for Netatmo information, see www.netatmo.com)
MIT License
27 stars 29 forks source link

Support For Energy in pyatmo #494

Closed tmenguy closed 4 months ago

tmenguy commented 7 months ago

Hi,

This one is the companion of https://github.com/tmenguy/netatmo_custom. All of this is to add energy measure support in homeassitant, so for pyatmo, extended handling of the /getmeasure endpoint. This one is not really up to date by the hour, from what I've seen it is updated very 3 to 4 hours, so I introduced an estimation using power and the measure to have a proper following of energy in homeassitant that is made to be "close to real time" .

Of course feel free to ask questions and so on. What I don't know is : is this one is working with the existing home assistant integration, it should, it is fixing some issues, but well, we never know, and I don't have a full HA setup for dev, more one for HACS.

BR Thomas

tmenguy commented 7 months ago

Thank you for your effort. I feel that this makes the code quite hard to follow and I am not sure if this solves the energy tracking properly. Netatmo has been quite sensitive to aggressive polling and I don't think we actually need this. No matter if you fetch data ever minute or every 10 seconds, it will be far off from realtime. Please correct me if I misinterpret your intention. Also I think this PR tries to solve too many things at once, which makes reviewing quite difficult.

Hi ... yes I do think you missed some stuff here :) and they are in the home assistant component: -there is the addition of the support of multi home support ... I'm not the only one with the need I think -previously the historical data and everything related to energy was really not working : as it was not supporting off peak / peak mode (very popular in France) -as for the polling (see the Netatmo component, it is not in the pyatmo lib of course): I actually made a big rework here as now the throttling is mathematically correct, and adapt gracefully with the number of publisher to be probed, it was mandatory as for energy you have to query each device individually (unfortunately the Netatmo API is made this way :( ) : I probe them every 30mn, but depending on the number of devices, I adapt this throttling to always comply with Netatmo given numbers (per hour and per 10s, according to their doc). I also very finely manage the exact number of calls per hour in a floating 1h window to be sure I stay in (and actually below )Netatmo recommendations, I've introduced also a proper handling of their throttling error if even fired, and a proportional/Derivative adaptation of the throttling. Also the energy APIs from Netatmo seems rather randomly being updated sometimes every 30mn, sometimes every 4hours, to help with that I also introduced and adaptation using the power data obtained to approximate the energy value during those non update period of time

All in all yes the code is more complexe but: -the throttling is a lot more robust and predictable than before (when due to the queue we could get a lot of time with no data) -the peak off peak model of Netatmo is a nightmare to handle, but we have it now -the energy measure are the best we can get with the state of Netatmo APIs today

I have a massive installation of Legrand devices (90 measure points) and it is working quite well, no throttling from the API (but well yes some timeout but it is different)

Hope it helps to understand the work here :)

BR Thomas

tmenguy commented 6 months ago

Small nudge :)