CharlesGillanders / homeassistant-alphaESS

Monitor your energy generation, storage, and usage data using the official API from Alpha ESS.
MIT License
101 stars 22 forks source link

Sunshine at night #127

Closed KortgatGH closed 2 months ago

KortgatGH commented 2 months ago

It may be coincidence, but since the previous update I get this every day:

image

The irregular energy reports coincide with false dips in solar production:

image

Any advice would be greatly appreciated.

Poshy163 commented 2 months ago

Can you confirm if these values (the 0) are shown in the logs or not? and see if its coming through the API (aka the information alpha is providing us)?

KortgatGH commented 2 months ago

@Poshy163 thanks for taking interest in this. I see no log entries in the system log that correlate with the times of these zero value anomalies. I also see no indication of any anomaly when looking at the Alpha data on the app or web portal. The separate PPV1 and PPV2 values on HA also looks normal. It seems to be only the cumulative solar production that dips

dhi20 commented 2 months ago

I've got the same issue. The debug log shows the following. It seems to be impacting every sensor which records a total kWh value (solar, charge, discharge, grid etc).

Logger: alphaess.alphaess Source: custom_components/alphaess/coordinator.py:54 integration: Alpha ESS (documentation, issues) First occurred: 12:01:00 AM (82 occurrences) Last logged: 9:28:29 AM

Unexpected json_response : {'code': 6026, 'msg': 'network is busy', 'expMsg': 'nested exception is org.apache.ibatis.exceptions.PersistenceException: \n### Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.\n### The error may exist in class path resource [mapper/td/SysColdataMapper.xml]\n### The error may involve com.alphaess.api.mapper.td.SysColdataMapper.selectOneDayLastPowerBysn\n### The error occurred while executing a query\n### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.', 'data': None, 'extra': None} when calling https://openapi.alphaess.com/api/getOneDateEnergyBySn?sysSn=ALDXXXXXXXXXX&queryDate=2024-09-02
Server disconnected
Error: Server disconnected when calling https://openapi.alphaess.com/api/getOneDateEnergyBySn?sysSn=ALDXXXXXXXXXX&queryDate=2024-09-02
Unexpected json_response : {'code': 6026, 'msg': 'network is busy', 'expMsg': 'nested exception is org.apache.ibatis.exceptions.PersistenceException: \n### Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.\n### The error may exist in class path resource [mapper/td/SysColdataMapper.xml]\n### The error may involve com.alphaess.api.mapper.td.SysColdataMapper.selectOneDayFirstPowerBysn\n### The error occurred while executing a query\n### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.', 'data': None, 'extra': None} when calling https://openapi.alphaess.com/api/getOneDateEnergyBySn?sysSn=ALDXXXXXXXXXX&queryDate=2024-09-02
Unexpected json_response : {'code': 6026, 'msg': 'network is busy', 'expMsg': 'nested exception is org.apache.ibatis.exceptions.PersistenceException: \n### Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.\n### The error may exist in class path resource [mapper/td/SysColdataMapper.xml]\n### The error may involve com.alphaess.api.mapper.td.SysColdataMapper.selectEntityList\n### The error occurred while executing a query\n### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.', 'data': None, 'extra': None} when calling https://openapi.alphaess.com/api/getSumDataForCustomer?sysSn=ALDXXXXXXXXXX
dhi20 commented 2 months ago

Rolling back to 5.02 seems to have fixed the issue. I tried 5.03 but it had the same issue.

Poshy163 commented 2 months ago

Its a combination of them giving invalid data (through their server errors here https://github.com/alphaess-developer/alphacloud_open_api/issues/82 and long time outs on requests), and us defaulting to using the value of 0 when receiving invalid data

They are aware of their server issues, but i can also see if i can work out a solution so instead of 0, it goes into some Unavailable state or something

beastien commented 2 months ago

I'm having exactly the same problem as dhi20. Last night it had happened over 60 times. I've tried using availability templates to make the sensors go unavailable to prevent the spikes, but then other sensors making calculations from those sensors get messed up because they don't receive valid data.

image

Poshy163 commented 2 months ago

Yeah, thats the issue at the moment, if one sensor, lets say solar production "fails" and is defaults to 0, all the other sensors that rely on it get incorrect info,

One thing we could do is somehow check the values, and if one of them is None we could just only return none, which iirc renders the data with the last accessible data-point (to the sensor)? will play around with it today/tomorrow

Poshy163 commented 2 months ago

The timeouts from the API on their end seems to have stopped in the last 10-20 mins? this shouldn't occur at all anymore

cabe1214 commented 2 months ago

It may be a little out of scope but any idea how to "clean-up" the wrong values in HA?

Poshy163 commented 2 months ago

its in the dev tools, in the statistics tab, i have linked the docs for you to check it out

https://www.home-assistant.io/docs/tools/dev-tools/#statistics-tab

KortgatGH commented 2 months ago

The timeouts from the API on their end seems to have stopped in the last 10-20 mins? this shouldn't occur at all anymore

@Poshy163 it seems on my side it still persists - this is for yesterday (2 September):

image

Poshy163 commented 2 months ago

Yeah, we have a solution in the draft PR linked above

beastien commented 2 months ago

Happened a couple of times again this afternoon.

image

beastien commented 2 months ago

And I'm getting this error in the log. 138 times in the last few days.

This error originated from a custom integration.

Logger: alphaess.alphaess Source: custom_components/alphaess/coordinator.py:54 integration: Alpha ESS (documentation, issues) First occurred: September 1, 2024 at 11:23:51 PM (138 occurrences) Last logged: 7:43:32 PM

Unexpected json_response : {'code': 6026, 'msg': 'network is busy', 'expMsg': 'nested exception is org.apache.ibatis.exceptions.PersistenceException: \n### Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.\n### The error may exist in class path resource [mapper/td/SysColdataMapper.xml]\n### The error may involve com.alphaess.api.mapper.td.SysColdataMapper.selectOneDayLastPowerBysn\n### The error occurred while executing a query\n### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.', 'data': None, 'extra': None} when calling https://openapi.alphaess.com/api/getOneDateEnergyBySn?sysSn=AL7011022080085&queryDate=2024-09-03 Server disconnected Error: Server disconnected when calling https://openapi.alphaess.com/api/getLastPowerData?sysSn=AL7011022080085 Unexpected json_response : {'code': 6026, 'msg': 'network is busy', 'expMsg': 'nested exception is org.apache.ibatis.exceptions.PersistenceException: \n### Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.\n### The error may exist in class path resource [mapper/td/SysColdataMapper.xml]\n### The error may involve com.alphaess.api.mapper.td.SysColdataMapper.selectEntityList\n### The error occurred while executing a query\n### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.', 'data': None, 'extra': None} when calling https://openapi.alphaess.com/api/getSumDataForCustomer?sysSn=AL7011022080085 Unexpected json_response : {'code': 6026, 'msg': 'network is busy', 'expMsg': 'nested exception is org.apache.ibatis.exceptions.PersistenceException: \n### Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.\n### The error may exist in class path resource [mapper/td/SysColdataMapper.xml]\n### The error may involve com.alphaess.api.mapper.td.SysColdataMapper.selectOneDayFirstPowerBysn\n### The error occurred while executing a query\n### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: td_alphacloud_alphaess - Connection is not available, request timed out after 10000ms.', 'data': None, 'extra': None} when calling https://openapi.alphaess.com/api/getOneDateEnergyBySn?sysSn=AL7011022080085&queryDate=2024-09-03

Poshy163 commented 2 months ago

Yeah, that's what i mean by the server issues. its a problem on their end unfortunately in the docs they just refer to a 6026 as a "internal error", but you can see in the response query its something to do with their database and it taking over 10s

Tons of people have tried reaching out regarding any fixes, but most of us have just got back lackluster info

CharlesGillanders commented 2 months ago

Tons of people have tried reaching out regarding any fixes, but most of us have just got back lackluster info.

I am coming to the conclusion that "ElevenFan" really doesn't have any autonomy, they are a developer that maintains the code that provides the public API but they don't appear to have any path to accept suggestions for the code nor to provide any support for the actual back end systems that the API is querying.

Their comment to contact after sales support is probably about as much direction as they can provide. We should find out how to correctly raise trouble tickets and then report issues with the API in exactly the same way as we would issues with the actual hardware.

beastien commented 2 months ago

Is it possible to make the values not go to zero when this happens as Poshy163 has mentioned? That's the main issue messing up the graphs and any calculations. If that's possible i think it wouldn't matter so much when these errors happen. I'm using availability templates at the moment but that still causes miscalculations when the sensors become unavailable.

Poshy163 commented 2 months ago

Yes there is, in https://github.com/CharlesGillanders/homeassistant-alphaESS/pull/130 it resolves it. im just testing it doesn't break anything else before taking the out of draft,

You can change the lines of code to fit what ive done, otherwise i plan to allow reviewing sometime in the next few hours, then Charles can bump version + release

Gaspode69 commented 2 months ago

Their comment to contact after sales support is probably about as much direction as they can provide. We should find out how to correctly raise trouble tickets and then report issues with the API in exactly the same way as we would issues with the actual hardware.

I guess in this case this was a misunderstanding. ElevenFan wrote that he fixed the issue in the OpenAPI a few hours ago. The affected values are delivered again.

However, the values are still not delivered resp. delivered with value 0!