async_update_data changes

[SVH-Powel]

I changed async_update_data to what is shown below. I am still testing!! (I know I should create a pull request for this, but that is not possible at this time and the computer I have available)

I know this will fail when going from summer- to winter time as one day will have 25 hours (or 23 in the spring, when changing from winter- to summer time)

`

async def _async_update_data(self) -> dict: """Get the latest data from ENTSO-e""" self.logger.debug("Fetching ENTSO-e data") self.logger.debug(self.area)

    time_zone = dt.now().tzinfo
    # We request data for yesterday up until tomorrow.
    yesterday = pd.Timestamp.now(tz=str(time_zone)).replace(hour=0, minute=0, second=0) - pd.Timedelta(days = 1)
    tomorrow = yesterday + pd.Timedelta(hours = 71)

    data = await self.fetch_prices(yesterday, tomorrow)

    parsed_data = self.parse_hourprices(data)
    data_all = parsed_data[-49:].to_dict()
    if parsed_data.size > 48:
        data_today = parsed_data[-49:-24].to_dict()
        data_tomorrow = parsed_data[-24:].to_dict()
    else:
        data_today = parsed_data[-24:].to_dict()
        data_tomorrow = {}

    return {
        "data": data_all,
        "dataToday": data_today,
        "dataTomorrow": data_tomorrow,
    }

`

[sfstar]

See #53 and the explanation in the original question thread #41

[stigvi]

See #53 and the explanation in the original question thread #41

Yes, of course :-) I saw a bug today that I wanted to fix. Today had yesterdays prices and tomorrow had todays prices. I suppose the intention is that both the today and tomorrow attribute list still change at midnight as they always did?

[stigvi]

I will test it until tomorrow afternoon and create a pull request when ready

[sfstar]

Yeah it's the same bug affecting #58 since they are computed by using self.data["dataToday"] Since it currently is only updated at the release of the new prices they are only calculated for half a usefull day at a time.

Perhaps it's a good idea to redesign the behaviour of the attributes and the average etc entities (They use the same datasource set). So that users can select whether they want the data to shift on:

• price publish (current scenario)
• rotation (daily at midnight changing the attributes and entities)
• sliding (updated every hour by removing previous hours that are no longer used).

[sfstar]

(rotation) For example I can think of people wanting to have everything reset when starting the new day.

(sliding) While for automation and planning only having the future hours to determine when to turn something on is better. Because this enables having a "consumption queue" that you fill with consumption tasks that need to be run. Having sliding values here is only logical since the queued thing needs to happen and previous pricing doesn't affect this need.

(price publish) Publish behaviour would be slightly different compared to the current behaviour. It will keep the "buggy" behaviour for the attributes while the average etc, entities would include today and tomorrow pricing from time of publish.

[sfstar]

If anybody can think of another behaviour type that somebody might want to have then please add to my list. The behaviour selection itself could easily be implemented by a options setting with an default (==price publish) and all the other options. Then the integration uses this option value in conditions for determining how to return attribute/entity data.

[sfstar]

@JaccoR could you (as the owner) say whether (if this change were to show up as an PR) you would be open to merging it? (Just checking so that no time is spend on something that will be still born 😳)

[JaccoR]

Nice idea, I am open for merging it if people are going to use it. I don't want to make the integration unnecessarily complicated though. I think the best way to handle this would be to add more attributes with all options. This way people who don't care and just want the sensors are not burdened with this "complicated" setting. I am not sure if this is possible though and if adding all those extra attributes causes any problems DB-wise. I currently have not a lot of time on my hands to work on this. Let me know if you are going to work on it and what your solution will be!

[sfstar]

My idea was to have an "advanced" option (located in or alike the seperate dialog for the pricing template). The option would be a predefined list (like the area selection) with an default that would use the current behaviour (so no simple users are affected). Then the integration would have an conditional check in the coordinator that checks which of the predefined list items was selected.

For example:

    def get_min_price(self, hourprices):
        return min(hourprices.values())

would become (psuedo code added to example function):

    def get_min_price(self, hourprices):
        if calculation.default:
            return min(hourprices.values())
        elif calculation.rotation:
            return min(hourprices current day checked) // will be updated at 00:00 every day
         elif calculation.sliding:
            return min(hourprices where hourprices is newer than date.now - timedelta 1h) // contains only current hour and newer

Of course it will be quite silly to have this check in every entity data function so it would most likely become a generic function that splits the hourprices array according to the set preferences.

If users want to have multiple behaviours of the integration (like for some automations they want default and for others sliding) they can just add an instance of the integration for each desired "advanced behaviour option".

The benefit for this over just including attributes would be that it is user configureable whether the DB space is taken up by the additional entities being provided. The setup would remain simple for existing and newbie users and the readme could explain the advanced selection and how to use it.

Since almost all users will have the desire to automate based on the data of this integration I believe that it is a "core"(although advanced) feature that this integration should have. For example currently I calculate sliding behaviour based on the price_list via node red. Since I'm not interested in past hours for making decisions about when to turn on a queued load. However, for other users they might just want to only turn something on if it's the cheapest hour in the last 48 hours.

example readme entry:

Advanced options

entity behavour (type - option list):

- Sliding
The min/max/etc entities will get updated every hour with only upcoming data.
This means that the min price returned at 13:00 will be the lowest price in the future.
Regardless of past hours that might have had a lower price (this is most useful if you want to be able to schedule loads as soon and cheap as possible)

- Default
The min/max/etc entities will get updated once new data becomes available.
This means that the min price will update once the next days pricing becomes available (usually between 12:00 and 15:00)
It also means that until the next days pricing becomes available the latest up to 48h of available data will be used to calculate a min price

- Rotation
The min/max/etc entities will get updated at midnight.
This means that the min price returned at 23:59 will  be based on the day x price while at 00:00 the day x+1 price will be the only one used in the calculations)
day x in this case is a random date like 2022-10-10

Based on my explaination would you agree that this is the preferred solution over adding attributes with the different behaviours?

Regarding implementation (if I get positive feedback on this proposal) I'm willing to create a PR. However, due to my current schedule it might take me a week (even with this being a relatively small change)

[sfstar]

71 is the implementation of the rotation / sliding / publish mechanisms

[JaccoR]

merged!

[sfstar]

Thank you, could you release the change? Also this issue can be closed now I guess :)