search

Ernst79 / degree-days

Integration that determines (weighted) degree days, gas consumption per degree day and a gas prognose for the current year based on weather info from the KNMI and your gas consumption.
MIT License
7 stars 3 forks source link

Prognosed gas not corresponding to mindergas.nl #18

Open Ernst79 opened 1 year ago

Ernst79 commented 1 year ago

Copied from #17 by @bakboord4

Noticed that the prognosed gas consumption shows about 60% what the dutch mindergas.nl website is prognosing. I have to find out how your integration is working. It would be helpful to me what is meant with 'year'. May I suggest to make in the documentation and in the sensor names everywhere explicit: calendar year (starting 1 jan) or billing year (date of year-gas-consumption is measured). Especialy for prognosed gas consumption. Is this for the whole billing year or to 'year to date' (today)? Thanks in advance for your reaction.

Ernst79 commented 1 year ago

@bakboord4 If I remember correct, it is based on a year starting from the date you have entered in the integration (the billing year.

image

I'm surprised that you get 60% of what mindergas.nl finds, it should be more or less the same (if you use the same settings in both mindergas.nl and this integration.

image

So, I also checked it, I have 902 m3 on mindergas.nl and about 800 m3 prognosed in the integration. Strange. I'm going to look into this.

bakboord4 commented 1 year ago

@Ernst79 your remark: 'I'm surprised that you get 60% of what mindergas.nl finds, it should be more or less the same (if you use the same settings in both mindergas.nl and this integration." I will check both settings.

bakboord4 commented 1 year ago

@Ernst79

My configuration. screenprint.pdf

Ernst79 commented 1 year ago

First checks.

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

month | dsmrreader | mindergas -- |-- | -- jan | 144 | 144 feb | 99 | 103 mar | 104 | 115 apr | 82 | 91 may | 20 | 24 jun | 9 | 10 total | 458 | 488

hmm, first have to investigate why they differ.....

The "meterstanden" are equal

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

datum | dsmrreader | mindergas -- | -- | -- 21-jun | 8597 | 8597 31-dec | 8109 | 8109 totaal | 488 | 488

Ernst79 commented 1 year ago

@Ernst79

My configuration. screenprint.pdf

Do you have the same settings on mindergas.nl?

image

Ernst79 commented 1 year ago

Het verschil in gasverbruik lijkt een bug in DSMRreader te zijn

https://github.com/dsmrreader/dsmr-reader/issues/1828 https://github.com/dsmrreader/dsmr-reader/issues/1811 https://github.com/dsmrreader/dsmr-reader/issues/1770

bakboord4 commented 1 year ago

@Ernst79 Thanks for settings remark. They have the same settings now. Also your "Parameters van het graaddagen-rekenmodel" is the same here. The difference is more or less the same. Gas prognose: 611,9 m3 en (minder gas:) 1074 m3 (expected total billing year consumption). 'My' billing year ends 15 august. Is billing year used for calculating 'gas prognose'? Unfortunately I am not using dsmrreader but HA integration DSMR Slimme Meter.

Ernst79 commented 1 year ago

it is based on the billing year. the prognose is calculated here, where the past weighted degree days are calculated on the actual KNMI data, and the future days (till the end of the billing year) are calculated on the average of the past 20 years (or at least, it should do that).

https://github.com/Ernst79/degree-days/blob/f63e5a66be6ebfd578370e5835766b195a4de148/custom_components/degree_days/knmi/__init__.py#L83

bakboord4 commented 1 year ago

@Ernst79 My lesson learned is to be sure that the Billing-YTD-gas-consumption in my test HA is recorded (entry in recorder.yaml). I am using a test HA remote connected to production HA giving the needed sensors.

Values. gas-prognose: 1055.7 m3 Billing-YTD-gas-consumption: 1055.864 m3 (mindergas) Prognose full billing year: 1074 m3 We are closing the gap!

This integration is running now for one hour or so. Due to the fact that the first two values are practically the same, I will wait until tomorrow to see what the results are of a possible night cycle and a bit more gas-consumption than zero in first hour of its operation. Expect (hope) to see that gas-prognose has narrowed the gap of 22 m3. W'll see.

bakboord4 commented 1 year ago

it is based on the billing year. the prognose is calculated here, where the past weighted degree days are calculated on the actual KNMI data, and the future days (till the end of the billing year) are calculated on the average of the past 20 years (or at least, it should do that).

https://github.com/Ernst79/degree-days/blob/f63e5a66be6ebfd578370e5835766b195a4de148/custom_components/degree_days/knmi/__init__.py#L83

Unfortunately I am not systems developer wring code. Some of the code give me an idea, most of it not. But the code is easy to read and is documented.

Ernst79 commented 1 year ago

has it been solved now?

bakboord4 commented 1 year ago

@Ernst79 No. The latest HA 2023.7.beta broke the remote home assistant on my testHA preventing connection to production DSMR gasmeter readings. There is still a difference between the gas-prognose of 1055m3 and MinderGas full billing year prognose of 1074m3 (15 August).
There is almost no difference between in MinderGas logged DSMR gasmeter and DSMR gasmeter in HA. I log in MinderGas the DSMR gasmeter value from HA. Have to wait for the new HA 2023.7 release to see if remote HA is fixed. When fixed I will try to find an explanation for prognose differences. Perhaps you have some suggestion for me.

bakboord4 commented 1 year ago

@Ernst79

Both instances of HA are up and running again. Below you will find a table with 2 days of values of DD integration compared with MinderGas website. Checked, the settings are the same. <html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">

YTD=7 en 8 juli 2023! Source DD7 juli MinderGas7 juli DD 8 juli Remarks:2 nights
Gas: verbruik afreken ytd DSMR-HA 1057,968 m3 1059,16 m3 1058,13 m3  
Gas: prognose DD/EK 1056,2 m3 1072 m3 10.00:1056,2 m3 00.00-10.00: 1053 m3
Gewogen graaddagen afreken jaar DD/EK 2463,88 WDD 2463,24 WDD 04.00:2467,8 WDD Update: 04.00
Gas: verbruik per gewogen graaddag DD/EK 0,3930 m3 0,398 m3 0,3930 m3  
           

 

Finding explanation for the difference was not possible. I read code but really understand the calculation was not possible for me. 1. Gas prognose. The values of DD and MinderGas are showing the biggest difference: 16 m3 more for MinderGas. The value of Gas Prognose (DD) is the almost the same for Gas consumption YTD (= verbruik, source calculated from DSMR). Is there an explanation possible? 2. The update cycle of DD integration is at midnight. However, Gas prognose is updated at 10.00 o'clock and WDD value at 4.00 o'clock. Is this performance as designed? 3. Gas-WDD values differ 0.004 m3, 1.825 m3 for an whole billing year. Is it correct to say it is negligible?
nelbs commented 1 year ago

I think the difference between the gas prognose values between DD (1056) and MG (1072) is very small. There are multiple ways to calculate the prognose so differences are inevitable. The most important factor for the calculation is the amount of historical data where the estimation is based on. DD uses for instance the data per day for the past 20 years. Changing the number of years can have a large effect on the prognose.

Note that the prognose becomes more accurate the closer you get to the end date.