[tibber] fetching "current price" on the hour (outside of the configured "refresh interval")

[binderth]

Your Environment

• Version used: openHAB 4.2.1-release (openHABian, raspberry Pi 4/4GB)

Problem statement

The "Current Total Price"-channel (current_total) refreshes with the configured "Refresh Interval" of the binding. Therefore it refreshes "randomly" - according to the setting and not "on the hour". If you need all the "cumulative" channels in a long interval it could be, that the current price is only refreshed every 6hours or 25minutes or so. The interval starts not "on the hour", but on the time, the binding is restarted and/or changed are made.

current behaviour

In my case, I wanted to have the not live-channels refreshed every 25minutes and therefore set the refresh interval accordingly. I did so on minute 3 of the hour, so the interval was started there: 3 - 28 - 53 - 8 - 31 - ... But you will never get the change "on the hour".

expected behaviour

The current-price is heavily reliant on (at least my) rules. I decide on the current price (and price level, ...) the behaviour of my high-power devices (wallbox, Heating, ...). So I relied on the value of the price is exactly the current one "on the hour". So, I'd like to have a refresh regardless of configuration "on the hour" of all releated channels, which provide information on the "current hour".

suggested solution

Perhaps the easiest would be to treat the intervall like cron: starting with 0 "on the hour" and then in the configured refresh intervall. In my case of 25 minutes: 0 - 25 - 50 - 0 - 25 - 50 -0 -... or to additionally refresh every hourly channel on the hour regardless of the configured refresh intervall. Channels I think should be refreshed on the hour are all the "non-live" ones, currently:

• current_total
• current_startsAt
• current_level
• hourly_from
• hourly_to
• hourly_cost
• hourly_consumption

bonus

I came across my prolonged refresh interval, because I got spamming events.log, described in https://community.openhab.org/t/oh4-2-tibber-binding-shall-process-timeseries-multiple-logentries/157290 So, as a bonus it would be great, if the binding only logged that INFO, if the price information changed, meaning the first time after 13:00h, that the prices change.

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/tibber-binding/67019/292

[lsiepel]

Why is it so important to have the channels refreshed on the hour? What is the use case?

[binderth]

Because the prices change on the hour? Use case is having the hour-related channels up-to-date? It's kinda useless, if you rely on "now it's low price level" or "now the current price is" values are updated minutes (23 mins in my example) later.

[kaikreuzer]

The use case is clear. In my case, I am using a refresh interval of 1 minute, so I never noticed an issue here.

I would think that if the new forecast persistence gets implemented, we will automatically see item state updates directly at the moment the value changes - which would make longer refresh intervals possible and this issue here would be solved in the most elegant way.

Is anybody by any chance working on forecast support or would know how to do it easily?

[jlaur]

Is anybody by any chance working on forecast support or would know how to do it easily?

I think you mean time series support? This was already implemented in #16275, so it should be possible to configure an item with forecast persistence strategy and have prices pushed to the event bus automatically as they become current.

I do still agree though that it would also be nice to have the binding update the current price when entering a new hour. This is of course redundant when having a persisted item configured with the forecast strategy. But OTOH it works out of the box without any persistence configuration, so it's convenient when not needing persistence (for other reasons), but just wanting to have the current price available.

[kaikreuzer]

I think you mean time series support?

Yes, that's what I meant. Heck, how could I have missed that this is already implemented? I was waiting for it all the time. 🤣

I do still agree though that it would also be nice to have the binding update the current price when entering a new hour.

Sure. The question is whether this should trigger a new bulk fetch of data or if it should just cache the value and push an update on the hour. I would possibly tend towards @binderth's suggestion to treat the refresh interval to always "start" at the full hour instead of the moment the handler is started up. Obviously, an initial fetch should be done nonetheless.

[jlaur]

Sure. The question is whether this should trigger a new bulk fetch of data or if it should just cache the value and push an update on the hour.

Separating data fetching and channel updating into two distinct tasks would be preferable IMHO, since it would result in fewer network calls. From openHAB's perspective this may not seem important, but it would still be good behavior not to have all clients asking for the same data over and over again at almost the exact same time (from the server's perspective).

I would possibly tend towards @binderth's suggestion to treat the refresh interval to always "start" at the full hour instead of the moment the handler is started up. Obviously, an initial fetch should be done nonetheless.

Sure, that would be a good first step, if not having a cache. Just have in mind that 15 minutes granularity is planned for Q1/2025, so potentially that would lead to 96 calls per day when actually only 1 call is needed in terms of fetching the new data.

[binderth]

Thanks for the discussion so far. Just a little heads up: With a little help of the forum here: https://community.openhab.org/t/tibber-binding-current-price-changes-minutes-half-an-hour-after-the-hour/158385/11 I found out, that

1. using the "forecast"-strategy on the Tibber Price item indeed let it change "on the hour"
2. this doesn't apply to the other items. I guess, they're not time series ready, e..g Price Level, hourly from/to (not sure, if anything but numbers allow for time series, though)

If the rules rely on the price level information (as mine do partly), it would still be super handy to get their values changed on the hour as I suggested. As for network calls, I also think a cache of sorts would do. on the other hand: do we know, that the 15minutes granularity still is auctioned off every 24hours? And that's also no changes in between? Perhaps that would be a way to go, to invalidate the cache around 13:00h and only let it fill, if there's data for the next day? Then there's the problem of how often the calls are fired: if they do on the "refresh interval", perhaps we should limit the number of minutes to lets say max 6hours or even less? There were days, when the next day prices were only published in the evening...

Leaving one thing: Should the refresh interval still be applied to all non-live channels? There's also the "hourly/daily consumption" channels. Valid use cases would be to get those updated in another interval as the day ahead fixed ones like current price, price level, startsAt, endsAt?

last question: I'd like to PR the documentation, so that it gets clearer, that setting up "forecast"-strategy leads to changes on the hour, independent of the refresh interval. Is it correct, that you have to have this strategy on the item for the default-persistence?

[jlaur]

2. this doesn't apply to the other items. I guess, they're not time series ready, e..g Price Level, hourly from/to (not sure, if anything but numbers allow for time series, though)

Correct, only channel current_total has time series support.

do we know, that the 15minutes granularity still is auctioned off every 24hours?

Yes, I believe nothing will change, except the granularity.

And that's also no changes in between?

Correct.

Perhaps that would be a way to go, to invalidate the cache around 13:00h and only let it fill, if there's data for the next day?

I don't know the Tibber binding well, and don't use it personally, but I can share my experience from working with the Energi Data Service binding.

Energi Data Service provides (amongst many other datasets) spot prices from Nord Pool. These prices are usually available at 13:00 CET (often a bit earlier). The binding fetches the day-ahead prices at 13:00 + up to one minute of jitter to not have all clients fetch them at exactly the same time. At this point a few things can happen:

• We now have prices for tomorrow. Next data fetch is scheduled tomorrow around 13:00.
• The call succeeds, but we don't have prices for tomorrow as expected, so apparently they are delayed. We now apply an exponential backoff strategy with some jitter. Next call in minimum 10 minutes, and maximum one hour between retries (i.e. we'll wait longer and longer between retries, but no longer than one hour).
• The call fails for any reason. If 429 ("Too many requests") we'll wait at least 30 minutes. Otherwise one minute with some jitter. In both cases exponential backoff.

When we finally have our day-ahead prices, they are cached for (at least) two reasons:

• To be able to perform fast calculations (implemented as Thing actions).
• To be able to update current price every hour. This job is scheduled with millisecond accuracy. This is redundant for channels persisted with forecast strategy, but important otherwise.

In this binding, prices can be received in either EUR or DKK. Since the prices are settled in EUR, the DKK values have been through a currency conversion. So after receiving the prices, they will not change, unless:

• The currency conversion is updated. I have seen very slight changes that can only be explained by this.
• Wrong prices were published by mistake. I have never experienced this. It has happened for the aWATTar binding though, because of what appears to have been a technical problem - see #17068.

So in general, there is no reason to call the service again until next day around 13:00.