Feature request: Stop updating prices at midnight

[SVH-Powel]

Please note that this is about the prices attribute. Not prices_today and prices_tomorrow

I wonder if it is possible to stop updating the prices attribute at midnight. I see no good reason for deleting half of the data at that time. The reason it makes things difficult is that I want to have the cheapest hours between 18:00 and 06:00 next day. But when prices updates at midnight and the previous day is deleted from the list, it is impossible to find the cheapest hours anymore. The history is gone and from midnight I can only find the cheapest hours from 00:00 to 06:00.

The template below works as expected from 18:00 to midnight, but at midnight the prices attribute changes and I get a new set of 6 cheapest hours. At the worst, I could have the 6 cheapest hours between 18:00 and 06:00 before midnight and get new 6 hours after midnight so that I get 12 hour in total.

I see no good reason to delete data from the prices attribute at midnight. A lot of automation happens at night, too, and changing this list when there is no changes in the data it contains, makes building templates a lot more difficult than it has to be. In fact, I dont know how to solve the problem in a template alone and has to use python scripting instead. I can handle that, but users with no experience in programming will struggle with making python scripts.

• platform: template sensors: cheapest_hours_between_18_and_06: value_template: >- {% set l=state_attr('sensor.current_electricity_market_price', 'prices')[18:30]|sort(attribute='price') %} {{ (now() >= as_datetime(l[0].time) and now() <= as_datetime(l[0].time) + timedelta(hours = 1)) or (now() >= as_datetime(l[1].time) and now() <= as_datetime(l[1].time) + timedelta(hours = 1)) or (now() >= as_datetime(l[2].time) and now() <= as_datetime(l[2].time) + timedelta(hours = 1)) or (now() >= as_datetime(l[3].time) and now() <= as_datetime(l[3].time) + timedelta(hours = 1)) or (now() >= as_datetime(l[4].time) and now() <= as_datetime(l[4].time) + timedelta(hours = 1)) or (now() >= as_datetime(l[5].time) and now() <= as_datetime(l[5].time) + timedelta(hours = 1)) }}

[JaccoR]

There has to be some moment past prices are purged and the prices attribute changes. This is now done at midnight, where yesterdays prices get purged and tomorrows prices become todays prices. This seemed the most convenient. I get your point, so maybe we can do this in a smarter way. Keeping the past prices for some extra time might solve your problem.

I am also thinking about dividing the attributes into individual sensors, so 48 sensors for every hour. This is in line with home assistant guidelines and won't hog that much DB space. How would that be handled by a template? This leaves you with the same problem i think.

[SVH-Powel]

Yes, 48 different sensors will be useless for me.

Regarding the update of the prices attribute, why not update it between 13:00 and 14:00 when new prices are available and only then?

[JaccoR]

It is not about the update moment (this happens every hour), but the moment previous prices get purged. When prices before 13:00 would be removed, graphs would only contain prices from 13:00 onward. This would not fix the issue, but would place it on another moment in the day: when you need the cheapest prices between 10:00 and 18:00 for example.

[SVH-Powel]

I think a prices attribute which always contain 48 elements and changes only when tomorrow prices are available, would be perfect. I cant see any good reason for deleting elements in this list at other times of the day. What is deleting data and shrink the list at other times going to solve?

Each day when tomorrows prices are available, prices[0:23] are set with the todays prices and prices[24:47] are set to tomorrows prices. Does not need to be more complex than that.

And it makes templates a lot easier to make when this list always have 48 elements and only updates a single time each day when there is a change in data from entso-e between 13:00 & 14:00

[ThorAlex87]

I seem to be facing this problem too, hopefully it's something that could be relatively easily solved at this relatively early stage.

[sfstar]

What about having the prices attribute always contain up to 72 hours worth of data. Meaning that every update at 00:00 would remove the data > 24h ago. Then you could also add an yesterday attribute containing yesterday's prices split from the list you already retrieved for the prices attribute. This would also allow users to more easily determine when to charge their battery. Since price hops would become more easily identifiable.

[SVH-Powel]

What about having the prices attribute always contain up to 72 hours worth of data. Meaning that every update at 00:00 would remove the data > 24h ago. Then you could also add an yesterday attribute containing yesterday's prices split from the list you already retrieved for the prices attribute. This would also allow users to more easily determine when to charge their battery. Since price hops would become more easily identifiable.

It would be better, but it will still update this attribute at midnight when there really is no changes in the data it contains. And if I understand you correctly, this list will still be variable in length and between 48 and 72 elements. That makes using it in Home Assistant templates more difficult than it has to be as elements change at midnight. You have to make some logic that can handle the list before midnight and another logic for after midnight.

[SVH-Powel]

The whole point for me was to have a list that does not change at midnight. As it is currently implemented, it is equal to the prices_today list for 14 hours and it change at midnigth as both prices_today and prices_tomorrow does. So the prices list could either be deleted or changed. It is redundant in its current state :-(

[ThorAlex87]

Could there be an option to choose preferred method when setting it up? "Update list at midnight" and "update list when new prices become available"?

[Mariusthvdb]

So the prices list could either be deleted or changed. It is redundant in its current state :-(

or only use that, and simply adjust the frontend. Agree on the many redundant attributes, for me though only using this entity is the temporary solution

(ofc, I also manually edited the attributes out except for this list)

[sfstar]

What about having the prices attribute always contain up to 72 hours worth of data. Meaning that every update at 00:00 would remove the data > 24h ago. Then you could also add an yesterday attribute containing yesterday's prices split from the list you already retrieved for the prices attribute. This would also allow users to more easily determine when to charge their battery. Since price hops would become more easily identifiable.

It would be better, but it will still update this attribute at midnight when there really is no changes in the data it contains. And if I understand you correctly, this list will still be variable in length and between 48 and 72 elements. That makes using it in Home Assistant templates more difficult than it has to be as elements change at midnight. You have to make some logic that can handle the list before midnight and another logic for after midnight.

What about retrieving up to 72 hours of data and in the integration pick the newest 48H to be put into the prices attribute. Then the prices attribute won't be just today and tommorow combined and it would update at 13:00 (when new prices become available) and always have 48 values to go through in templating etc.

[JaccoR]

this has been implemented!

[SVH-Powel]

this has been implemented!

It has? In which version? I am using 0.1.1 and it is not implementet there.

[SVH-Powel]

Saw now that you have started with releases and this was implemented in main and ready for the next release.

[sfstar]

@SVH-Powel it is implemented in #53 (and some bugfixes in commits to main after that). @JaccoR has made alot of quality of life improvements like this but hasn't made a release for them yet. I'm cheating a little bit by overwriting the custom_component files with the latest copy of main, but I'm also looking forward towards a new x.x.2 release :)

[JaccoR]

There were still some bugs in the main copy, so I was waiting to release 0.1.2. They are fixed now though, so 0.1.2 is coming shortly!

[sfstar]

Thank you so much for releasing :)