Failed startup due to corrupted datapoint

[ToonDeb]

Describe the bug The integration didn't start up, because one of the calls failed:

/v1/chargeSessions/summary/users/****?&startTs=1546300800000&endTs=1706745599000&granularity=MONTH
with an error {"timestamp":1704291796131,"status":500,"error":"Internal Server Error","message":"Mapped Measurements were empty! Likely because wh was null. ","path":"/v1/chargeSessions/summary/users/****"}

I think it is related to an error on the back-end of ohme, in the app the charge sessions graph doesn't load correctly for some months, which leads me to think there is some error on their back-end somewhere, because of corrupted data.

I managed to fix it myself by setting the startTs to a more recent value, 1704236400000, which caused the call to succeed.

To Reproduce I think the error is in their back-end, not something that can be reproduced locally.

Expected behavior Maybe a better error handling can help resolve this? Something else I was thinking about was updating the summary to always retrieve the current day and use the "utility meter" type in home-assistant, to not have a potentially intensive query to ohme every time the call is done?

Screenshots

2024-01-03 14:23:15.559 DEBUG (MainThread) [custom_components.ohme.api_client] GET request to /v1/users/me/account, status code 200
2024-01-03 14:23:15.602 DEBUG (MainThread) [custom_components.ohme.api_client] GET request to /v1/chargeSessions, status code 200
2024-01-03 14:23:15.603 DEBUG (MainThread) [custom_components.ohme.coordinator] Finished fetching Ohme Charge Sessions data in 0.042 seconds (success: True)
2024-01-03 14:23:15.642 DEBUG (MainThread) [custom_components.ohme.api_client] GET request to /v1/users/me/account, status code 200
2024-01-03 14:23:15.643 DEBUG (MainThread) [custom_components.ohme.coordinator] Finished fetching Ohme Account Info data in 0.040 seconds (success: True)
2024-01-03 14:23:16.140 DEBUG (MainThread) [custom_components.ohme.api_client] GET request to /v1/chargeSessions/summary/users/dSy8z3nNJuVyTOVOCZKQUjO3Vii2?&startTs=1546300800000&endTs=1706745599000&granularity=MONTH, status code 500
2024-01-03 14:23:16.140 ERROR (MainThread) [custom_components.ohme.api_client] Ohme API response error: /v1/chargeSessions/summary/users/dSy8z3nNJuVyTOVOCZKQUjO3Vii2?&startTs=1546300800000&endTs=1706745599000&granularity=MONTH, 500; {"timestamp":1704291796131,"status":500,"error":"Internal Server Error","message":"Mapped Measurements were empty! Likely because wh was null. ","path":"/v1/chargeSessions/summary/users/****"}
2024-01-03 14:23:16.141 DEBUG (MainThread) [custom_components.ohme.coordinator] Finished fetching Ohme Charger Statistics data in 0.498 seconds (success: False)
2024-01-03 14:23:21.771 DEBUG (MainThread) [custom_components.ohme.api_client] GET request to /v1/users/me/account, status code 200
2024-01-03 14:23:21.817 DEBUG (MainThread) [custom_components.ohme.api_client] GET request to /v1/chargeSessions, status code 200
2024-01-03 14:23:21.818 DEBUG (MainThread) [custom_components.ohme.coordinator] Finished fetching Ohme Charge Sessions data in 0.045 seconds (success: True)
2024-01-03 14:23:21.860 DEBUG (MainThread) [custom_components.ohme.api_client] GET request to /v1/users/me/account, status code 200
2024-01-03 14:23:21.862 DEBUG (MainThread) [custom_components.ohme.coordinator] Finished fetching Ohme Account Info data in 0.043 seconds (success: True)
2024-01-03 14:23:22.370 DEBUG (MainThread) [custom_components.ohme.api_client] GET request to /v1/chargeSessions/summary/users/****?&startTs=1546300800000&endTs=1706745599000&granularity=MONTH, status code 500
2024-01-03 14:23:22.370 ERROR (MainThread) [custom_components.ohme.api_client] Ohme API response error: /v1/chargeSessions/summary/users/****?&startTs=1546300800000&endTs=1706745599000&granularity=MONTH, 500; {"timestamp":1704291802360,"status":500,"error":"Internal Server Error","message":"Mapped Measurements were empty! Likely because wh was null. ","path":"/v1/chargeSessions/summary/users/****"}
2024-01-03 14:23:22.371 DEBUG (MainThread) [custom_components.ohme.coordinator] Finished fetching Ohme Charger Statistics data in 0.509 seconds (success: False)

Home Assistant Version 2023.12.1

Integration Version 3.0.0

ohme Go (the stick one, in the middle of the cable

Additional context If i have some time this weekend I might take a look myself and create a pull request

[ToonDeb]

Thanks for the integration by the way, it works great for me besides this!

[dan-r]

That is strange - not something I've come across.

Were you able to identify where the corrupted datapoint was? I'd be interested to see if it was before or after you first installed the Ohme.

The utility meter method would definitely be a better way of implementing that sensor. I'm going to be away for a week from this weekend so won't have a chance to look into it for a while (except for maybe adding some better error handling). If you get a chance to have a look in the meantime, contributions are always appreciated! :)

[dan-r]

@ToonDeb I've put out v0.3.2-beta.1 which 'tolerates' setup failures for coordinators that we could feasibly live without.

I know it doesn't fix the issue for you and if anything makes it worse than your own changes but it seems a bit silly to completely fail setup because we can't get a value for one sensor.

I've added the CT reading endpoint to the same optional list, but the rest are all somewhat interlinked so should still cause a setup failure.

[dan-r]

@ToonDeb have you had a chance to look at this or had any more issues with the integration starting? It should tolerate errors fetching certain data now.

[dan-r]

What happened? The Accumulative Energy Usage sensor is marked as unavailable in HA and has no data:

The usage data is available in the Ohme app:

Expected behavior The energy usage to match the data available in the Ohme app.

Log Output Please note I have replaced what looks like a user token with a “USERID” string below:

Logger: custom_components.ohme.api_client
Source: custom_components/ohme/api_client.py:108
Integration: Ohme (documentation, issues)
First occurred: February 3, 2024 at 10:34:58 PM (25 occurrences)
Last logged: 10:05:24 AM

Ohme API response error: /v1/chargeSessions/summary/users/USERID?&startTs=1546300800000&endTs=1709251199000&granularity=MONTH, 500; {"timestamp":1707033920882,"status":500,"error":"Internal Server Error","message":"Mapped Measurements were empty! Likely because wh was null. ","path":"/v1/chargeSessions/summary/users/USERID"}

Home Assistant Version 2024.1.5

Integration Version 0.5.1

Additional context Other sensors work fine. Charger is an Ohme Home Pro. Happy to help fix/debug.

Behaviour seen again

[dan-r]

@stuartmacd I've closed your issue and quoted you here as it appears to be the same issue. Can you see stats for past months properly in the ohme app?

[stuartmacd]

Thanks @dan-r, agree this looks like the same issue. I can scroll through charging sessions back to Oct 2021 (when the charger was commissioned). Tomorrow I'll try setting the start timestamp nearer that date as a workaround for now.

[dan-r]

Interesting - yeah I'd suggest moving that start timestamp in api_client.py until I come up with a better fix... I believe there is a commisioning date somewhere in the API that I could use

[dan-r]

@stuartmacd turned out to be easier than expected. Would you mind trying v0.5.2-beta.5? (beta instructions).

[stuartmacd]

Thanks but no luck I'm afraid:

This error originated from a custom integration.

Logger: custom_components.ohme.api_client
Source: custom_components/ohme/api_client.py:109
Integration: Ohme (documentation, issues)
First occurred: 9:45:44 PM (2 occurrences)
Last logged: 9:45:48 PM

Ohme API response error: /v1/chargeSessions/summary/users/USERID?&startTs=1634083200000&endTs=1709251199000&granularity=MONTH, 500; {"timestamp":1707083144710,"status":500,"error":"Internal Server Error","message":"Mapped Measurements were empty! Likely because wh was null. ","path":"/v1/chargeSessions/summary/users/USERID"}

The timestamp looks ok, that is the day of commissioning, the app shows data for that date (but not before):

[dan-r]

How very strange... Would you mind testing out and seeing the earliest timestamp that works for you?

[stuartmacd]

Yes leave it with me for now and I'll figure it out this week. Thanks again for the help.

[stuartmacd]

I did a binary search over the chargeSessions API and concluded that it doesn't return data specifically from 13th March 2023. Dates before that are good, dates after that are good.

I also discovered that the Ohme app doesn't show cumulative stats for March 2023 claiming "We couldn't get the stats for the selected period" (even though the detailed charge sessions breakdown does show that day). I've raised the app issue to Ohme but given it's not crippling the app, I'm not particularly hopeful.

I value this sensor to track usage day-to-day, but don't necessarily need "all energy ever used". Could I perhaps contribute a separate "previous day energy usage" sensor? Very open to suggestions.

[dan-r]

Interesting. Might be worth contacting Ohme about that as it seems like something weird going on their end. There's a 'session energy usage' sensor now that might be useful?

The purpose of the all usage sensor was to be used with the energy dashboard in Home Assistant - not sure how that works with the session one...

[dan-r]

I've deprecated this sensor and hidden it behind a config option in the latest beta, so will close this down for now unless someone else wants to give a go at it. I don't see much demand for it especially with the session energy usage sensor now.