Support for Nordpool

[marval]

Let me start by saying this is not complete, but I assume a lot of Barry users might be interested in this as the company is closing.

Nordpool provides raw electricity prices for a lot of Nordic countries - Norway, Sweden, Denmark, Finland, etc. This price would not be the final price incl. taxes and tariffs. I am working on something similar to the HomeAssistant integration of Nordpool where one could specify a template for calculating the extra charges. While this is under development you might be able to just use a flat rate using FeePerKilowattHour.

PS: Danish customers getting the taxes refunded might get pretty close to the actual price using this without anything else.

Tests

I have tested this on my own data and if I try to calculate the total price incl. taxes I get quite a difference, but this is probably due to me having a HVAC heating the house and thus reduced taxes. Barry puts all the taxes for 13 kWh in the first hour of the day (typically charging my car then) and then the rest is close to the raw el. price. I would love to get someone else data to try and test it. Here are my results:

[MattJeanes]

Looking good so far, nice work! Just to let you know I've just disabled C# nullability in the solution as it's a pain and doesn't outweigh the benefits, so the issues you're seeing in the build should be solved now if you merge back in from my main branch.

You could potentially expand the FeePerKilowattHour to take a percentage as well as a number to allow more advanced scenarios there too which could solve things like VAT cost e.g. +20% on top of Nordpool pricing, should be a simple and effective change.

[marval]

Thanks @MattJeanes! Will look into merging main (as I should have done before opening PR, but ...).

The HomeAssistant integration allows for some very special template parsing where they kind of have their own scripting going on. This would be out of the scope of my implementation,. It is very similar to the FixedPriceprovider so I might look into combining it together with Nordpool somehow.

[MattJeanes]

I only just made that change 10 minutes ago so you wouldn't have picked it up yet anyways 😁

I am looking to make some big improvements to FixedPrice soon as it's a bit too limited at the moment, such as for #22

[marval]

Do you have an ETA on that part? I might look into enabling multiple providers in price calculation and then just depend on your newly developed functionality?

[MattJeanes]

I don't unfortunately it's just whenever I find some time, I hope to do it soon though. Could you link me to the HomeAssistant stuff you mentioned? Might be useful!

[oivindoh]

@marval nice stuff! Which endpoint are you hitting at Nordpool for this? I can't see the appSettings entry for NP

[PalleRaa]

Great to see some progress here. Let me know if you need some testing on these early releases.

[PalleRaa]

Any chance I can get this version @marval ? What do I put in my docker-compose to get this version?

If you could get the Nordpool spot price with vat (multiply their price with 1.25) the rest of the price for transport, tax etc is very constant (except for the 17-20 timeslot, but who charges here anyway) and can just be added using the FeePerKilowattHour

[marval]

Hi @PalleRaa Yeah .. would try to find some time to get it done by the end of the weekend. Family, work, C19 ... need more hours per day :D

[marval]

So I somehow managed upload my test DB password and Barry API key ... not of big importance but shameful :(

I believe I have cleaned up some debugging and testing stuff and the code is now ready for review. Hopefully @oivindoh and @PalleRaa would soon be able to try it out.

[PalleRaa]

I would love to try it out, but as you know I am complete noob. What should I put in my docker-compose.yaml to fetct this nordpool build? I have the following, but it does not work when I run docker-compose up

  teslamateagile:
    image: marval/teslamateagile:nordpool
    restart: always
    environment:
    - DATABASE_USER=teslamate
    - DATABASE_PASS=secret
    - DATABASE_NAME=teslamate
    - DATABASE_HOST=database
    - TeslaMate__UpdateIntervalSeconds=30
    - TeslaMate__GeofenceId=4
    - TeslaMate__EnergyProvider=Nordpool
    - Nordpool__Currency=DKK # See below allowed currency codes by Nordpool
    - Nordpool__Region=DK2 # See below a list of all Nordpool regions
    - Nordpool__VAT=1.25 # VAT multiplier. In this example 25%
    - Logging__LogLevel__Default=Debug

[oivindoh]

Looks great!

I built oivindoh/teslamateagile:nordpool-arm64-v2 from the marval:nordpool branch, deleted all my charging costs for my home id (e.g. update charging_processes set cost=NULL where geofence_id=1; in psql and run a deployment with this config tacked on:

# <DB-settings>
TeslaMate__UpdateIntervalSeconds: "300"
TeslaMate__EnergyProvider: Nordpool
TeslaMate__GeofenceId: "1"
Nordpool__Currency: NOK
Nordpool__Region: Tr.heim
Nordpool__VAT: "1.25"

The majority of my charging sessions update to what looks like valid cost numbers, e.g.

 id |         start_date         |        end_date         | charge_energy_added | charge_energy_used | cost
----+----------------------------+-------------------------+---------------------+--------------------+-------
 19 | 2022-03-07 15:44:42.976    | 2022-03-07 21:25:18.724 |               29.40 |              32.82 |  5.90`

I get issues with some of the charge sessions, like these:

 id |         start_date         |        end_date         | charge_energy_added | charge_energy_used | cost
----+----------------------------+-------------------------+---------------------+--------------------+-------
 13 | 2022-02-23 19:51:45.412    | 2022-02-24 07:33:27.002 |               60.48 |              67.91 |
 22 | 2022-03-16 22:45:42.898    | 2022-03-17 06:29:08.532 |               39.97 |              44.77 |
 26 | 2022-03-23 22:35:01.388    | 2022-03-24 05:26:48.255 |               33.32 |              39.74 | 

What they all seem to have in common is that they're crossing day boundaries. Here's the log for session 13:

info: TeslaMateAgile.PriceHelper[0]
      Calculating cost for charges 02/23/2022 19:51:45 UTC - 02/24/2022 07:33:27 UTC
[
  {
    "Value": 0.1468875,
    "ValidFrom": "2022-02-24T00:00:00+00:00",
    "ValidTo": "2022-02-24T01:00:00+00:00"
  },
  {
    "Value": 0.1387250,
    "ValidFrom": "2022-02-24T01:00:00+00:00",
    "ValidTo": "2022-02-24T02:00:00+00:00"
  },
  {
    "Value": 0.1361000,
    "ValidFrom": "2022-02-24T02:00:00+00:00",
    "ValidTo": "2022-02-24T03:00:00+00:00"
  },
  {
    "Value": 0.1381000,
    "ValidFrom": "2022-02-24T03:00:00+00:00",
    "ValidTo": "2022-02-24T04:00:00+00:00"
  },
  {
    "Value": 0.1511500,
    "ValidFrom": "2022-02-24T04:00:00+00:00",
    "ValidTo": "2022-02-24T05:00:00+00:00"
  },
  {
    "Value": 0.1619375,
    "ValidFrom": "2022-02-24T05:00:00+00:00",
    "ValidTo": "2022-02-24T06:00:00+00:00"
  },
  {
    "Value": 0.1727250,
    "ValidFrom": "2022-02-24T06:00:00+00:00",
    "ValidTo": "2022-02-24T07:00:00+00:00"
  },
  {
    "Value": 0.179375,
    "ValidFrom": "2022-02-24T07:00:00+00:00",
    "ValidTo": "2022-02-24T08:00:00+00:00"
  }
]
fail: TeslaMateAgile.PriceHelper[0]
      Failed to calculate charging cost / energy for charging process 13
      System.Exception: Charge calculation failed, pricing calculated for 1328 / 2056, likely missing price data
         at TeslaMateAgile.PriceHelper.CalculateChargeCost(IEnumerable`1 charges) in /src/TeslaMateAgile/Helpers/PriceHelper.cs:line 123
         at TeslaMateAgile.PriceHelper.Update() in /src/TeslaMateAgile/Helpers/PriceHelper.cs:line 66

[oivindoh]

@PalleRaa if you run on arm64/raspberry pi or similar, you could always use my image.

If not, building is easy enough; something like this (replace bullseye-slim-amd64 with bullseye-slim-arm32v7 if running arm32):

git clone https://github.com/marval/TeslaMateAgile.git
cd TeslaMateAgile
git checkout nordpool
docker build -t teslamateagile:local-nordpool -f TeslaMateAgile/Dockerfile . --build-arg=bullseye-slim-bullseye-slim-amd64

Then just use teslamateagile:local-nordpool as your image in docker-compose

[oivindoh]

@marval I changed the GetPriceData() method in NordpoolService.cs to naively fetch the "from" day of data in a separate request and loop over responses if from and to days are not the same. I get the expected results:

 id |         start_date         |        end_date         | charge_energy_added | charge_energy_used | cost
----+----------------------------+-------------------------+---------------------+--------------------+-------
 13 | 2022-02-23 19:51:45.412 | 2022-02-24 07:33:27.002 |               60.48 |              67.91 | 10.22
 22 | 2022-03-16 22:45:42.898 | 2022-03-17 06:29:08.532 |               39.97 |              44.77 |  7.39
 26 | 2022-03-23 22:35:01.388 | 2022-03-24 05:26:48.255 |               33.32 |              39.74 |  6.32

My implementation here probably leaves a lot to be desired:

    public async Task<IEnumerable<Price>> GetPriceData(DateTimeOffset from, DateTimeOffset to)
    {

        var urls = new List<string>();

        if (from.Date != to.Date)
        {
            // Naively add a request for <from> day if from and to are not the same day.
            // Breaking assumption would be charging sessions spannning several days
            urls.Add(_options.BaseUrl + "?currency=," + _options.Currency + "&endDate=" + from.UtcDateTime.ToString("dd-MM-yyyy"));
        }

        urls.Add(_options.BaseUrl + "?currency=," + _options.Currency + "&endDate=" + to.UtcDateTime.ToString("dd-MM-yyyy"));

        List<Price> prices = new List<Price>();

        foreach (String url in urls)
        {
            var resp = await _client.GetAsync(url);

            resp.EnsureSuccessStatusCode();

            var nordPoolResponse = await JsonSerializer.DeserializeAsync<NordpoolResponse>(await resp.Content.ReadAsStreamAsync());

            if (nordPoolResponse.data.Rows.Count > 0)
            {
                foreach (NordpoolResponseRow row in nordPoolResponse.data.Rows)
                {
                    if (row.IsExtraRow)
                    {
                        continue;
                    }

                    foreach (NordpoolResponseRowColumn column in row.Columns)
                    {
                        decimal value;
                        Decimal.TryParse(column.Value.Replace(',', '.'), out value);

                        var area = column.Name;

                        if (_options.Region == area
                            && row.StartTime >= from.AddHours(-1)
                            && row.EndTime < to.AddHours(1)
                            && !prices.Any(x => x.ValidFrom == row.StartTime)
                        )
                        {
                            prices.Add(new Price
                            {
                                ValidFrom = row.StartTime,
                                ValidTo = row.EndTime,
                                Value = (value / 1000) * _options.VAT
                            });
                        }
                    }
                }
            }
        }

        return prices;

    }

[PalleRaa]

Wow. I actually managed to build "my own" container from @marval code using the short guide made by @oivindoh . It now works and updates prices. I added a FeePerKilowattHour=0.728 which is what I use when manually calculating the price of charges. However there seems to be something wrong with the time zones somewhere. I.e. I have charged 2022-03-28 from 22:00 to around 23:30 DK Time. However in the teslamateagile log I see:

Calculated charge cost for 03/28/2022 20:00:00 UTC - 03/28/2022 21:00:00 UTC 
(unit cost: 0.00, fee per kWh: 0.728): 7.85649077546666672451840 for 10.79188293333333341280 energy

Calculated charge cost for 03/28/2022 21:00:00 UTC - 03/28/2022 22:00:00 UTC 
(unit cost: 0.7405875, fee per kWh: 0.728): 8.395011986973166447592574000 for 5.71638529333333318416 energy

So in one case it fetched a cost of 0 which is obviously wrong. In the time it charged from 23:00 - 23:30 local time it fetches the Nordpool price from 21:00-22:00.

I understand I am clearly not your level of programming expertise so please ignore my questions/findings if they are too trivial and caused my noob mistakes by me.

[PalleRaa]

@marval and @MattJeanes is there any chance this is being worked on and can be integrated in the official branch? Since Barry closed down I need to calculate my own prices in excel for each charge session before entering it into TeslaMate

[marval]

@PalleRaa will look into the issues and try to boot up the project again. I am on spot pricing again so the incentive is there for me too :)

[lauer]

I got the current version working - however, I could not get it to read the FeePerKilowattHour env variable to add the fixed extra cost per kwh.

[MattJeanes]

@lauer try changing the env var name to TeslaMate__FeePerKilowattHour

[PalleRaa]

@marval maybe it is easier to get the price from https://www.energidataservice.dk/guides/api-guides since their API is well documented. Unfortunately is does not support as many price regions as Nordpool.

[marval]

Just a quick update - I started working on a solution using EnergiData's API and FixedPrice (to accommodate for tariffs and taxes). Will update you once I got something working.

[lauer]

Great - another idea could be integrating https://github.com/rndfm/elspotpris (https://elspotpris.dk/) So you can define the company you buy the electricity from, so you get the correct price.

[oivindoh]

Integrating through an intermediary like elspotpris or energidataservice rather than the source of the data seems like a poor approach. A better approach would then be to use entsoe, which covers all of Europe, and is considered an official price formation source.

[marval]

I might look into Entsoe/Nordpool again but want to implement EnergiData to be able to compare

I also thought about Elspotpris but they have no API. MinStrøm is more or less closed by a payment so it is a no-go. Creating an API for a similar service is achievable, but would require saving the data (to avoid throttling by the APIs it would call) and so on. Not binning the idea completely but not now.

I am hopimg to have something for the few Danish users by end of this week and then work on Nordpool during the Christmas holiday.

On Thu, 15 Dec 2022 at 16.15, Øivind Hoel @.***> wrote:

Integrating through an intermediary like elspotpris or energidataservice rather than the source of the data seems like a poor approach. A better approach would then be to use entsoe, which covers all of Europe, and is considered an official price formation source.

— Reply to this email directly, view it on GitHub https://github.com/MattJeanes/TeslaMateAgile/pull/27#issuecomment-1353249507, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALAOD5XSPR2JWI6GDLUPYTWNMYY3ANCNFSM5PA6XGFA . You are receiving this because you were mentioned.Message ID: @.***>

-- /Martin

[marval]

@PalleRaa @lauer @oivindoh see #34

[jysaloma]

Is this workable in main or do you need to use local repository? Atleast documentation is not.

[marval]

Hi @jysaloma, No, it does not work in the main branch since I moved away to Energinet which is only for the danish market. Nordpool was kind of hard to work with and as far as I remember they could require payment. If you know their API endpoints and limits I am willing to put a bit of work in making it work.

[jysaloma]

Well, this approach seems to work decent. https://github.com/MattJeanes/TeslaMateAgile/pull/27#issuecomment-1082329611

This integration is widely used in Home automation if that could be use as API endpoint example. https://github.com/custom-components/nordpool