Reconciliation of gaps in Glow API / incorrect cumulative values

[leesuttonuk]

Describe the bug Not certain if this is a bug with how the API is being consumed (in HA) or if it's a bug in the Glow API itself...

Had a couple days of missing readings; this morning no gas / electricity readings between 04:00-10:00am. Likewise, when checking at the time this was not showing in the Bright app. So possibly a DCC data delay.

This has now returned to the Glow app, reconciling with my (non-Glow) CAD (Chameleon) readings. But hasn't pulled through into Home Assistant through the integration. Nor via influxdb into Grafana with the daily/yearly sensors.

So there's a gap and the cumulative value at 10am picked up from the 4am value. No outage with my HA Container as all other sensors behaved as expected during this period.

To Reproduce Steps to reproduce the behaviour:

1. Identify a period of no data in the Bright app / via Home Assistant
2. Data eventually populates in Bright app
3. Data gap then exists and doesn't show in Home Assistant

Expected behaviour Data to correct itself cumulatively in the event of gaps in upstream data consumption from Glow (and upstream of Glow to DCC)

Screenshots From Grafana on the sensor:

Both today/year for electricity show no usage

Glow app confirming readings:

Version Integration: Latest (v0.5.2 tag) HA Container: 2022.2.3

Debug log N/A - was not enabled at time, have enabled now for future occurrences

Additional context Restarted HA server and no difference

[ColinRobbins]

It the data is not showing I the Brigt App, then there is nothing this integration can do.

The data flow is…

• Your meter to DCC
• DCC to Hildebrand (visible in Bright app)
• Hildebrand to HA.

[leesuttonuk]

Correct @ColinRobbins Though, see my screenshot confirming it's then populated in the Bright app but not via integration.

[ColinRobbins]

Hildebrand have been having issues recently, and the API is not reliable. After 3-4 days reliable working, i have a gap from 5am to 10am today for example. What do you HA log files say?

[leesuttonuk]

Thanks @ColinRobbins times correlate so looks like it was an upstream API issue then. I didn't have debug enabled at the time though have since enabled, and everything since has caught up.

Does the integration have the ability to catch it up once the API regains itself? Or, does the gap just exist permanently?

2022-02-13 14:16:58 DEBUG (SyncWorker_1) [custom_components.hildebrandglow_dcc.glow] get 2: (https://a pi.glowmarkt.com/api/v0-1/resource/{uuid}/readings?from=2022-02-13T00:00:00&to=2022-02-13T23:59:59&period=P1D&offset=0&function=sum)

[ColinRobbins]

It catches up - on each successful call, the API returns the number of kWh used since the start of the day/year.

[leesuttonuk]

Does the catchup take a day then? As - as shown in my screenshots there's just the gap and the first successful cumulative reading picks up from where it started failing.

[ColinRobbins]

It all depends on where the failing was. If Hildebrand API it should be in an hour or two. If between Hildebrand and DCC, or DCC and your meter, then I’m not sure. The “catchup” function to request latest data from DCC is called every 30 minutes. Eventually, my understanding is it should come right.

[HandyHat]

Unfortunately there is no way to backdate data in Home Assistant, which means that when looking at the recorder/Grafana, the time when the API wasn't returning any values will forever show no readings. The integration does ask for the total value for the day/year, which means that the total value for the day/year will retroactively update and be accurate, but this increase will be attributed to the wrong time in the day.