BottlecapDave / HomeAssistant-OctopusEnergy

Unofficial Home Assistant integration for interacting with Octopus Energy
https://bottlecapdave.github.io/HomeAssistant-OctopusEnergy/
MIT License
570 stars 57 forks source link

Home mini electricity data no longer updating every minute #914

Closed paulmthomas2 closed 3 months ago

paulmthomas2 commented 3 months ago

Describe the bug

sensor.octopus_energy_electricity_REDACT_current_demand (linked to the Home mini) is no longer updating each minute. The API would generally seem to be working as Octopus App and octoaid are showing live data, but this sensor seems only to be picking up new values c. every 30 minutes

Reproduction steps

ongoing

Expected behaviour

expect updates every minute (sensor is currently calling every minute still, but not returning new data)

Tariff Code

E-1R-INTELLI-VAR-22-10-14-H

Integration Version

11.1.1

Home Assistant Version

2024.6.2

Fresh Install?

Not specified

Home Assistant Logs

home-assistant_octopus_energy_2024-06-14T15-40-18.911Z.log

Confirmation

paulmthomas2 commented 3 months ago

OK - this seems to be up and running again, so will close

paulmthomas2 commented 3 months ago

actually this is back behaving the same way again - only updating every 30 mins.

stevenbrown1862 commented 3 months ago

I've also noticed this today. It may have coincided with me updating home assistant.

paulmthomas2 commented 3 months ago

I've also noticed this today. It may have coincided with me updating home assistant.

I updated HA Core today, but only after I noticed this recurring, and it hasn't made a difference!

Kyleidge commented 3 months ago

Yup same issue here. Although it happened after a local powercut...

pcb1962 commented 3 months ago

I'm seeing half-hourly updates from the Home Mini since 2pm today. Home Pro and Glow are reporting normally.

image

martinpelant commented 3 months ago

Same for me - today afternoon it started updating only once every 30 minutes - I didn't change or update anything when that happened. Since then I updated HA core and this integration but to no effect

pantherale0 commented 3 months ago

I noticed the same issue today. No errors in logs relating to this integration.

image

BottlecapDave commented 3 months ago

Hello and sorry that you're seeing this issue. I haven't personally seen this. Can you keep track of the data_last_retrieved attribute? This should update at the frequency that you've specified. If it is, then it's suggesting an issue with the API not updating the data. If this isnt the case, there should be warnings in your logs around using cached data.

Another thing to bare in mind is if you're accessing the data in other ways (e.g. octowatch) as this will count towards your hourly limit which is 100 times an hour for the specific API across all meters.

en5ads commented 3 months ago

Hello, just to say this is happening to me too, about 14:00 yesterday and hasn't resolved yet. I don't have any other methods of accessing the data. Anthony

JimRnewell commented 3 months ago

Same for me too since around 2pm yesterday

Kev1507 commented 3 months ago

Same is happening for me too, also since 2pm yesterday. The data_last_retrieved attribute is updating approximately every minute though but the actual reading updates every half an hour.

lepkirk commented 3 months ago

Same for me, since 2pm yesterday (17/6/24) I started getting updated only every 30 mins. I haven’t made any changes to my system or anything octopus since weeks ago.

gwilford commented 3 months ago

I'm monitoring two separate accounts with minis and both are now only updating infrequently within HA.

pcb1962 commented 3 months ago

My Last Retrieved time is updating every minute as configured, I'm not running any other Octopus apps, and I've unplugged my Home Pro - no change, must be an issue at the Octopus end.

paulmthomas2 commented 3 months ago

just to add to my comments above, i'm still getting (when i choose to look) updates every minute on the Octo-Aid app, which is driven by the API. I don't have anything else that auto-calls on the API, so it's not an API limiting matter as far as I can see.

plord12 commented 3 months ago

Not much to add, but I'm seeing the same. For me started at 2pm yesterday ... maybe related to when I updated to 11.2.0 ?

Screenshot 2024-06-18 at 11 21 21
BottlecapDave commented 3 months ago

I'll have to investigate this manually when I get a chance. The integration is just reporting back the data that comes from the API and the fact that this seems to have effected a variety of users across a variety of versions of the integration at around the same time suggests OE have changed something on their end. This might be limited to how I'm requesting the data (which is why other things like Octowatch aren't effected) or they're doing something to requests coming from this integration (which would not be helpful with no heads up).

smeg9 commented 3 months ago

+1 for the 2PM yesterday

I updated to the latest version today, before that I was a few versions behind.

image
BottlecapDave commented 3 months ago

I've raise this within the forums so awaiting a response.

townsmcp commented 3 months ago

Just want to add a +1 for me as well. The data_last_retrieved attribute is updating approximately every minute though but the actual reading updates every half an hour. Also only using Octo-Aid. I did have the Glow IHD working for near realtime reporting right up until saturday when I had 5 back to back power cuts over the course of an hour which seems to have blown up the IHD/smart meter comms hub :(

smsm1 commented 3 months ago

+1 I'm also having the issue from about 2pm on 17th June 2024. With so many having the issue it does suggest a change in Octopus Energy and.

funks81 commented 3 months ago

I’m also having this issue - have tweeted @agilephil at Octopus to see if this is an issue with Octopus.

wralb commented 3 months ago

I have the same symptoms starting precisely 2PM on 17th. My Home Assistant (except the DB) is in its own VM so I rolled back to a snapshot from the 16th. The issue was still there with this snapshot; this appears to rule out local software of config changes, and points to a change in the behaviour of the OE API.

For info... My OE App is still providing minute by minute data. By comparing this to hass, it seems the data being provided by OE is the consumption for one minute every 30 minutes (not the average over those 30 minutes); so the data is particularly misleading now.

funks81 commented 3 months ago

So - I've had a reply from @agilephil - he doesn't suggest any problems with the Octopus API - but has suggested the issue may be the API rate limit. I find this a little strange - as I thought the rate limit was 100 per hour - and I've only ever requested 1 minute updates - and has been working fine for the last year or so.

paulmthomas2 commented 3 months ago

So - I've had a reply from @agilephil - he doesn't suggest any problems with the Octopus API - but has suggested the issue may be the API rate limit. I find this a little strange - as I thought the rate limit was 100 per hour - and I've only ever requested 1 minute updates - and has been working fine for the last year or so.

It seems odd that it would be the API rate limit, bearing in mind that (should I choose to), i can watch the same minute-frequency updates in Octo-Aid (and that's still working). Note that i don't habitually do this, so I don't think it's an issue with overall API calls because i'm using them with another device/system.

smsm1 commented 3 months ago

I wonder what API call OctoAid is using it is still working there?

BottlecapDave commented 3 months ago

I don't have access to the source code for octoaid, but my guess is it's doing demand and consumption separately or requesting the deltas at a granular level. Granular deltas is an option, but I'm worried about people restarting their HA instances closer to midnight that could result in up to 1440 records being returned instead of the current max of 48. To minimise the input I would need to cache the results between restarts, and be smart so I don't write too much to disk.

funks81 commented 3 months ago

I noticed the same issue today. No errors in logs relating to this integration.

image

So - when I look back at my history - mine seems to have stopped working at exactly the same time as yours - just after 1400 on the 17th. Not sure if this is relevant/exactly the same for others?

perdimia commented 3 months ago

This has also started happening on ours the last couple of days

sopski8890 commented 3 months ago

+1 for us too. Only updating every 30 mins. Last evaluated and last retrieved updates every minute as per config, but value doesn't update

Seaniau commented 3 months ago

+1 Same issue, same start time. 17th June 14:00.

sdjmchattie commented 3 months ago

Same on ours as well:

image

Last value change on the minute was at 14:01:56 the 17th on our HA instance, then it switched to half hours where the value changes even though it does get pulled every minute as usual.

Other anomalies on our historic data are me resetting the network, where the line has a break and, weirdly, one instance of the value changing twice in successive minutes around 16:01.

@BottlecapDave you mentioned before that you hadn't seen this personally. Is that still the case? Just trying to understand if this is associated with our account/tariff/usage.

Rifraferty commented 3 months ago

I noticed the same issue today. No errors in logs relating to this integration. image

So - when I look back at my history - mine seems to have stopped working at exactly the same time as yours - just after 1400 on the 17th. Not sure if this is relevant/exactly the same for others?

I got the same issue which occurred just after 2pm on the 17th too

Nothing on my account or tariff has changed

wralb commented 3 months ago

The environment where I'm seeing this issue:

Devices: Home Mini & Ohme Home Pro EV Charger - nothing else accessing my account I am aware of exchept very infrequent use of Octopus app and web site. Tarrifs: E-1R-INTELLI-VAR-22-10-14-J & G-1R-LOYAL-FIX-12M-24-02-24-J HASS: bottlecapdave integration v11.2.0 on hass 2024.6.3 in python 3.12.4 venv on ubuntu LTS 22.04.4 in VM on XCP_NG 8.3 hypervisor

Rolling back to snapshot prior to 17 June didn't help. Setting the query interval to 5 mins instead of 1 min doesn't help.

townsmcp commented 3 months ago

Does anyone have access to the forums at https://forum.octopus.energy/login ? Maybe others have been discussing similar regarding API's? I have just sent them an email as per the page

paulmthomas2 commented 3 months ago

image

I've just observed that as of sometime between 17th June and end of 18th June, the cumulative consumption sensor also started only returning new information every thirty minutes (attempted to put a history graph screen shot in - showing the smooth curve on 17th with minute-by-minute data points, and then a lumpy one on 18th - i've cut it to try to avoid showing meter numbering data)), which suggests this is more widespread in terms of how data is being returned from the API?. it also means (as it seems to come just after the half and and hour), that I effectively lose the last half hour's consumption as it then resets to nil for start of the next day.

BottlecapDave commented 3 months ago

@sdjmchattie nope I'm seeing it as well. It feels like they changed the behaviour of how the demand is reported in a grouped scenario. I want data grouped by half hour for consumption sensors, but want the latest demand for the demand sensor. This used to work fine, but the demand now seems to be an average over the grouped period. I'd make two calls if it wasn't for the hard request limit on this API, which the integration is already pushing.

I'm still waiting for a response, but if I have time over the weekend I'll start looking at making delta calls with finer grouping (which will probably be worse for their servers in scenarios where a user is at the end of the day that needs the whole days worth of data for some reason (e.g. no internet all day)

JuzzzyP commented 3 months ago

I think this may now be resolved fingers crossed just started updating every 60 seconds or so. Thanks for the fine work!

wralb commented 3 months ago

Confirm that around 8.46 today I started getting more regular data, seems to be working as designed again. Fingers crossed.

smsm1 commented 3 months ago

Same here that it's suddenly started working again.

JimRnewell commented 3 months ago

same here around 8:47am this morrning

pcb1962 commented 3 months ago

Improved but not resolved, I'm currently seeing updates at 2 minute intervals. image

wralb commented 3 months ago

Updating every minute here... Screenshot_20240621-093855_1

JimRnewell commented 3 months ago

yes also every minute here

perdimia commented 3 months ago

Mine is also back to updating one a minute

BottlecapDave commented 3 months ago

Well if that's fixed, it will save me a job over the weekend :). They responded this morning saying they'll check with the relevant teams so maybe they've reverted whatever they were doing before?

BottlecapDave commented 3 months ago

@pcb1962 the time might not change if the value of the demand doesn't change between retrieval of data (HA thing beyond my control), as it looks like you have some updates that are 1 minute apart.

plord12 commented 3 months ago

Yep, fixed for me as well (just as the home pro dropped through the letterbox!)

townsmcp commented 3 months ago

Fixed here as well