Communication to Senec blocked

axel19640815 commented 3 months ago

With which version of the integration you have an issue?

SENEC.Home V3 hybrid/SENEC.Home V3 hybrid duo/SENEC.Home V2.1 (or older)

Multiple Systems Environment?

[ ] I have multiple SENEC Systems (master/slave configuration)

Checklist

[X] I have installed the latest release (or BETA) version of the integration and home assistant.
[X] I have prepared DEBUG log output (for technical issues) | In most of the cases of a technical error/issue I would have the need to ask for DEBUG log output of the integration. There is a short tutorial/guide 'How to provide DEBUG log' here
[X] I understand the SHIT-IN - SHIT-OUT principle (in case of implausible data) | It will happen, that the data that is displayed by this integration does not make much sense (to you) - Of course this could be always the case, that something in this integration has been messed up - but so far - in all reported issues the root cause of implausible data was/is, that the SENEC.Home device itself already provided this data
[X] I confirm it's really an issue | In the case that you want to understand the functionality of a certain feature/sensor Please be so kind and make use if the discussion feature of this repo (and do not create an issue) - TIA

Add a description

I am a newby using Home Assistant and tested the SENEC Integration. it works fine and gives me what I want to see. But as soon as I run it and it communicates with my SENEC battery, the regular communication from the battery to SENEC cloud is disturbed. I get notifications via the my-SENEC portal, that the battery was disconnected from the Internet. The effect is reversible. When I stop HA, the failure with the SENEC Cloud disappears immediately. Is it about too much traffic? Then, how can I reduce the requests from HA to a minimum? I played already with the refresh time but with almost no effect. is the communication traffic depending on the number of entities used or activated? Or does it ask always every information?

any idea? I should not be the only one having that issue…

Add your DEBUG log output

No response

marq24 commented 3 months ago

when you increase the pull frequency in the settings - you might need to restart the integration so that the change/adjustment will have an effect... I run mine here with 5sec - without any issues [or at least I did not notice any - since I do not have any SEBEC apps installed...

But using 60sec as update Interval should be save... at least that was the default value once I took over the development (created this branch here)...

ETH1166 commented 3 months ago

Just a confirmation: Since a few days I am facing the same issue. Senec's communication seems to be disturbed. Since today HA does not receive updated generated solar energy from the Webinterface. Before the integration worked flawlessly for several months. I updated to the latest plugin version. Problems still remain. My update rate on the web interface is set to 300 seconds. Thus the update rate does not seem to be the issue... Any advise is very much appreciated. Best regards Thomas

HA: Generated Energy is flat since one day:

marq24 commented 3 months ago

just in case - how to change the update-freqency...

The "flat" Generated Energy line could come from the fact, that @ SENEC the clock sometimes runs backwards, and - for what ever reason - the values provided are smaller then previous ones - in this case the integration ignore the data provided via the WebAPI... since we do not have the super power to travel back in time...

ETH1166 commented 3 months ago

Since 25.06. afternoon also the Senec Android App goes nuts:

For me it looks like the data is not delivered continously or correctly from my Senec box to the Senec servers.

In contrast to @axel19640815, who writes

I get notifications via the my-SENEC portal, that the battery was disconnected from the Internet. I do not see a similar warning on my Senec system, that there is a problem to send data to Senec.

The mein-senec.de application shows "good" data (or seems to filter out bad data):

In summary: It looks like a problem of Senec again!

marq24 commented 3 months ago

I know that does not help - here (for me) all is still fine with SENEC and SENEC data... Screenshot_20240628-083655

vmpr commented 3 months ago

same here, my data from the senec_webapi is not accurate anymore - maybe they changed the API-endpoints with an update or it is maybe a temporary problem. the data in the senec app is still accurate. a bit of a shame as I am using the data from the webapi for my energy dashboard with the total values in kwh - hopefully the will fix the problem soon

marq24 commented 3 months ago

IF there would had been a change in the API-Endpoints, then all users "must" be effected in the same way [except they failed to deploy the new version on all instanced and/or fucked up the setup of the Loadbalancer]...

At least I am not affected (yet/this time?) - which is a indication that is has to do with specific accounts (only) - I have seen this in the past, where "my data" was corrupted - while others had no issue at all... So let's wait and see :-/

vmpr commented 3 months ago

interesting update: my webapi entities just started to collect data again :)

Screenshot 2024-06-28 at 15 05 30

like u said, lets see what happens, its 3pm here in Perth / Australia :)

donbcd commented 3 months ago

I have the same problem with the flat line: The solar generated power is like that since two days, other values were properly updated until 2h ago, when the "Netz Bezug" started not updating anymore. In the Senec App, everything looks fine.

Update: I found this in the debug logs:

2024-06-28 11:20:31.764 DEBUG (MainThread) [custom_components.senec.sensor] Thanks for nothing Senec! prev>new for key powergenerated_total - prev:6415.96 new: 6408.81
2024-06-28 11:20:31.764 DEBUG (MainThread) [custom_components.senec.sensor] Thanks for nothing Senec! prev>new for key gridexport_total - prev:3650.66 new: 3632.76

So this really looks like Senec is giving me the wrong numbers? And it is always the same value (6408.81) for hours/days... So, is the App using a different API? Because the App shows the correct values?

marq24 commented 3 months ago

So this really looks like Senec is giving me the wrong numbers? And it is always the same value (6408.81) for hours/days... So, is the App using a different API? Because the App shows the correct values?

Have in mind, that there is no official API from SENEC - so the things I do with the WabAPI Integration is always a "fishing in the dark"... [mainly the call's generated via the mein.senec.de web portal]

ETH1166 commented 3 months ago

It looks like data is again coming from Senec:

The 12:00 peak is of normal height of about 4 kWh for a single hour and not representing the missing three days (would be about 100 kWh in my case).

In case of HA problems, where ha is not receiving data, the next reading would be the cumulated power for the whole period.

Thus, the data seems to be lost, which is a contradiction to Senec's App, which partly has data. The issues of the App, as described above, still remain...

Thank you again, @marq24 for your great support!

coro1404 commented 3 months ago

same problem here - long periods of smaller data - started two days ago out of nowhere

marq24 commented 3 months ago

the hint from @donbcd will help you a lot to identify the root cause of the problem, if the log shows the 'Thanks for nothing...' message, then you can be sure, that there is no technical issue with the communication between HA and Senec Web - it's just that the provided data just sucks.

coro1404 commented 3 months ago

the hint from @donbcd will help you a lot to identify the root cause of the problem, if the log shows the 'Thanks for nothing...' message, then you can be sure, that there is no technical issue with the communication between HA and Senec Web - it's just that the provided data just sucks.

sadly confirmed:

2024-06-30 21:56:10.051 DEBUG (MainThread) [custom_components.senec.sensor] Thanks for nothing Senec! prev>new for key consumption_total - prev:13774.06 new: 13767.26 2024-06-30 21:56:10.051 DEBUG (MainThread) [custom_components.senec.sensor] Thanks for nothing Senec! prev>new for key powergenerated_total - prev:16083.26 new: 16046.68 2024-06-30 21:56:10.051 DEBUG (MainThread) [custom_components.senec.sensor] Thanks for nothing Senec! prev>new for key accuimport_total - prev:2522.25 new: 2521.42 2024-06-30 21:56:10.051 DEBUG (MainThread) [custom_components.senec.sensor] Thanks for nothing Senec! prev>new for key accuexport_total - prev:2637.16 new: 2634.19 2024-06-30 21:56:10.051 DEBUG (MainThread) [custom_components.senec.sensor] Thanks for nothing Senec! prev>new for key gridexport_total - prev:7494.34 new: 7467.1

coro1404 commented 3 months ago

as i cannot see that from the debug log: are there more values returned by the API like timestamp (hence the assumption that senec is travelling back in time)?

AndreWohnsland commented 3 months ago

I once again also have the same issue, that senec returns wrong values. Sadly, as @marq24 mentioned, there is not much to do about it then wait until the data is "correct". This may be that the data was not correct before (it was sending 10 but was in reality 12) and now is correct (sending 12), or any other reason, whatever Senec is using causing this.

The API returns an increasing counter (it should always increase, it is something like the mileage of your car, or the total produced/consumed energy in the lifetime of the plant) but sometimes it can happen that senec will send 11 > 11.1 > 11.1 > 20 > 20 > 11.2, which would completely throw off HA calculation, because this is a monotonic increasing value. In case the new value is smaller than the old stored value, this integration will log this event and not update to this jumped value.

So, hopefully, their API is soon once again working properly 😓

VolkerMal commented 3 months ago

since 11:35am CEST today SENEC WEB API does delivery again the values. I see a significant increase on: sensor.senec_webapi_powergenerated_total sensor.senec_webapi_gridexport_total Seems all missing/non-delivered data since June 30th came in at once today at 11:35am.

For the sake of clarity - I had not done any modification on the SENEC intrgration in HA. Seems to me , SENEC had an issue which got fixed today.

During the entire time, SENEC APP provided reasonable and correct data.

ETH1166 commented 3 months ago

Hello @VolkerMal

since 11:35am CEST today SENEC WEB API does delivery again the values. I see a significant increase on: sensor.senec_webapi_powergenerated_total sensor.senec_webapi_gridexport_total Seems all missing/non-delivered data since June 30th came in at once today at 11:35am.

I can confirm this: 11:48 there was a jump of about 100 kWh (equivalent of 3 cloudy days for my small system) and now solar production seems to climb up monotonically again. Nevertheless, during the last days data had been on and off for several hours each...

During the entire time, SENEC APP provided reasonable and correct data.

Please try to switch the time resolution to 5 Minutes. In this graph I find drops to zero, potentially coming from missing data. When switching to 1 hour resolution, those dropouts disappear. I am wondering, whether the readings are correct though.

I am considering to switch to Riemann sums and abandon the SENEC WebAPI. Any thoughts on this idea in the context of the current problems are more than welcome. I cannot imagine Riemann sums giving less trustworthy data compared to a Webinterface, which is potentially loosing data sets.

Best regards Thomas

VolkerMal commented 3 months ago

Hello @ETH1166 (Thomas), after switching to the 5min resolution view in the SENEC app, I have no drops at solar production during the relevant time.

Concerning the Riemann sum function, I tend to agree to your thinking. I started to use the energy dashboard based on Riemann Trapez and it worked pretty well. Of course, you will experience very small deviations from Riemann sum function versus SENEC WEB API data. I also measured for two months or so all three solutions Riemann left, right and trapez. there are marginal deltas only. Well, someone wrote in an older treatd, a benefit of the SENEC WEB API data is, that no data is lost in case your HA is not running as the WEB API holds all the data even the Internet connection from the SENEC battery to the SENEC Cloud was down for a certain amount of time. That might be true, but I cannot confirm nor negate.

The only reason for me to chose the WEB API solution was to have exactly the same figures in my HA dashboard and the SENEC APP. But IMHO there is no reason not to go for Riemann, at least my HA runs much more stable than the SENEC app & data and the local network connection between HA and SENEC Home V3 never experienced any issues.

best regards, Volker

axel19640815 commented 2 months ago

My original problem was obviously not about not getting correct data from the SENEC cloud, but my battery CPU communication too busy when serving HA plugIn and SENEC at the same time. The version of my battary is quite old.

So, the question is, if there is an opportunity to reduce traffic from the HA to the battery in order to not keep the Battery CPU occupied. I can see, that reucing the interval of request influences the nummber of failures reported by the SENEC cloud, but at times below 60 sec it does not send data to the cloud at all.

Has anyone similar experience?

marq24 commented 2 months ago

since my Battery is in FERNABSCHALTUNG since a couple of days [Battery completely switched OFF remotely by Senec (for what ever reason???] I can't assist here in this subject...

marq24 commented 2 months ago

any news/updates on this issue? - if not I will close this - since there is not much the integration code can do... (it's just the messenger)...

marq24 / ha-senec-v3