search

MattJeanes / TeslaMateAgile

Integration to automatically fill in prices for charge data captured by TeslaMate for smart energy providers
MIT License
82 stars 10 forks source link

Can't calculate certain fixed prices #20

Closed think-free closed 3 years ago

think-free commented 3 years ago

Issue repeat everyday around this time interval : 2021-06-22 23:00:23.707 | 2021-06-23 00:04:17.124 (UTC in database which is 2021-06-23 01:00:23.707 | 2021-06-23 02:04:17.124 local)

Other calculations are ok ...

`

I have thoses prices defined :

`

MattJeanes commented 3 years ago

Hey @think-free, does it work if you change the 24:00 to 00:00 on that last one?

See the guide on the readme on how to regenerate charge costs if you're not sure.

Thank you for providing the output logs, very helpful!

think-free commented 3 years ago

Hi, thanks for the quick response. No, same error ... Current charge cost for theses rows (23:00 UTC, 01:00 local) is NULL, others are calculated successfully (manually setted all the costs for the current month to NULL for testing previously)

think-free commented 3 years ago

Interesting ...

  - FixedPrice__TimeZone=UTC
  - FixedPrice__Prices__0=22:00-06:00=0.15
  - FixedPrice__Prices__1=06:00-08:00=0.2
  - FixedPrice__Prices__2=08:00-12:00=0.3
  - FixedPrice__Prices__3=12:00-16:00=0.2
  - FixedPrice__Prices__4=16:00-20:00=0.3
  - FixedPrice__Prices__5=20:00-22:00=0.2

-> Works fine Problem with day change ? 23:00 UTC is the day before 01:00 local

MattJeanes commented 3 years ago

Interesting indeed! Looks like it could be an issue around timezones (argh!!)

Would you mind exporting one of the failed charges as a CSV so I can replicate the issue locally?

I'll need the data from the charges table for a particular charging_process_id.

This data contains no sensitive information like location but I recommend checking it yourself before sending over to verify.

Feel free to email me directly if you do not want to share it publically, you can find my email address on my GitHub profile.

think-free commented 3 years ago

Sent by mail

MattJeanes commented 3 years ago

Thank you but I need the data from the charges table not the charging_processes table (although that also does help!)

paulhargreaves commented 3 years ago

Installed yesterday. And I have the same problem, but I'm Europe/London.

      Updating prices
info: TeslaMateAgile.PriceHelper[0]
      Calculating cost for charges 06/23/2021 23:30:08 UTC - 06/23/2021 23:39:27 UTC
fail: TeslaMateAgile.PriceHelper[0]
      Failed to calculate charging cost / energy for charging process 4
      System.Exception: Charge calculation failed, pricing calculated for 0 / 28, likely missing price data
         at TeslaMateAgile.PriceHelper.CalculateChargeCost(IEnumerable`1 charges) in /src/TeslaMateAgile/Helpers/PriceHelper.cs:line 106
         at TeslaMateAgile.PriceHelper.Update() in /src/TeslaMateAgile/Helpers/PriceHelper.cs:line 54
info: TeslaMateAgile.PriceService[0]
      Waiting 300 seconds until next update
  teslamateagile:
    image: mattjeanes/teslamateagile:latest
    restart: always
    environment:
      - DATABASE_USER=teslamate
      - DATABASE_PASS=secret
      - DATABASE_NAME=teslamate
      - DATABASE_HOST=database
      - TeslaMate__UpdateIntervalSeconds=300
      - TeslaMate__GeofenceId=2
      - TeslaMate__EnergyProvider=Octopus
      - Octopus__RegionCode=B # Octopus Energy only
      - TeslaMate__EnergyProvider=FixedPrice
      - FixedPrice__TimeZone=Europe/London
      - FixedPrice__Prices__0=04:30-00:30=13.8
      - FixedPrice__Prices__1=00:30-04:30=4.5

Not sure how to generate a CSV for the table, but this should be what you need. charges-dump.txt

MattJeanes commented 3 years ago

Many thanks for the data! I'll look into this and let you know.

dakaix commented 3 years ago

Also having the same issue here, Europe/London as well.

MattJeanes commented 3 years ago

As it turns out I'm actually experiencing this issue myself now as I've recently switched off Agile (those prices!) to Go. I will look into this as soon as I can, it's been a busy couple of weeks sorry!

The good news is that as soon as the fix is deployed TeslaMateAgile will be able to pick up and calculate the costs which are failing currently without any user intervention (apart from updating TeslaMateAgile of course!)

jrothlis commented 3 years ago

charges.txt

Same issue here -- just switched from Agile to Go.

- TeslaMate__EnergyProvider=FixedPrice
- FixedPrice__TimeZone=Europe/London # IANA (tz database) time zone code, used for below times
- FixedPrice__Prices__0=00:30-20:30=15.9
- FixedPrice__Prices__1=20:30-00:30=5

@MattJeanes a couple of questions:

Sorry for all the questions!

MattJeanes commented 3 years ago

@jrothlis it is indeed in pennies, I will clarify documentation to show this soon

I'm not sure if having a geofence cost set would work with TeslaMateAgile as it waits for TeslaMate to mark it as completed at which point it would likely have already set a cost which would cause TeslaMateAgile to ignore the charge.

Octopus' API does indeed work historically as far back as it began I believe so if you set the various charges cost to null and set TeslaMateAgile back to agile settings it should work fine to recalculate them 🙂

MattJeanes commented 3 years ago

I stand corrected, it is actually in pounds (or your currencies normal unit aka euros / dollars etc), oops! I will definitely correct documentation to clarify this (£15/kWh anyone?!) I will leave it as it is in the code to avoid breaking changes, you can tell I didn't use FixedPrice myself!

I've managed to find some time tonight to investigate and it turns out there were 2 separate bugs, both timezone related (of course) which didn't rear their head until daylight savings time showed up. I've fixed them and added additional test coverage to cover these new scenarios, hopefully haven't broken anything in the process I've got a fair amount of tests on FixedPrice for this very reason!

I have published release candidate v1.7.0-rc.3, can you all please update your setups to use this version and let me know if it solves your problem? You will need to explicitly use this version if you are using the latest/no tag in docker as pre-release versions do not update the latest tag.

It has correctly calculated 3 charges on my own setup which were previously failing (though I need to re-calculate all of them as they're 100x more expensive than they should be 😳)

Sorry for all the disruption, hope this solves everything!

jrothlis commented 3 years ago

Thanks @MattJeanes I appreciate you spending the time!

Unfortunately it still seems to be not fixed on my end (my charge psql data is linked in my earlier comment). I updated the docker-compose.yml with image: mattjeanes/teslamateagile:v1.7.0-rc.2 and it pulled down a new image so I think that part worked ok, i.e. no error (I don't know how to check which version I am running). I updated my prices to 0.159 and 0.05 (and I also updated my logging settings as per your comments on the other issue, thanks!), and still get an error:

root@teslamate [ /var/lib/teslamate ]# docker-compose logs teslamateagile
Attaching to teslamate_teslamateagile_1
teslamateagile_1  | 2021-07-04 12:57:38.676 info: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Price service is starting
teslamateagile_1  | 2021-07-04 12:57:38.680 info: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Using energy provider FixedPrice
teslamateagile_1  | 2021-07-04 12:57:38.681 info: Microsoft.Hosting.Lifetime[0]
teslamateagile_1  |       Application started. Press Ctrl+C to shut down.
teslamateagile_1  | 2021-07-04 12:57:38.682 dbug: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Updating prices
teslamateagile_1  | 2021-07-04 12:57:38.682 info: Microsoft.Hosting.Lifetime[0]
teslamateagile_1  |       Hosting environment: Production
teslamateagile_1  | 2021-07-04 12:57:38.682 info: Microsoft.Hosting.Lifetime[0]
teslamateagile_1  |       Content root path: /app/
teslamateagile_1  | 2021-07-04 12:57:38.858 info: TeslaMateAgile.PriceHelper[0]
teslamateagile_1  |       Updating prices
teslamateagile_1  | 2021-07-04 12:57:40.124 info: TeslaMateAgile.PriceHelper[0]
teslamateagile_1  |       Calculating cost for charges 06/22/2021 19:38:30 UTC - 06/22/2021 23:34:58 UTC
teslamateagile_1  | 2021-07-04 12:57:40.257 dbug: TeslaMateAgile.PriceHelper[0]
teslamateagile_1  |       Calculated charge cost for 06/22/2021 19:30:00 UTC - 06/22/2021 23:30:00 UTC (unit cost: 0.05, fee per kWh: 0): 1.4428218574305555737425 for 28.85643714861111147485 energy
teslamateagile_1  | 2021-07-04 12:57:40.258 fail: TeslaMateAgile.PriceHelper[0]
teslamateagile_1  |       Failed to calculate charging cost / energy for charging process 238
teslamateagile_1  |       System.Exception: Charge calculation failed, pricing calculated for 619 / 630, likely missing price data
teslamateagile_1  |          at TeslaMateAgile.PriceHelper.CalculateChargeCost(IEnumerable`1 charges) in /src/TeslaMateAgile/Helpers/PriceHelper.cs:line 106
teslamateagile_1  |          at TeslaMateAgile.PriceHelper.Update() in /src/TeslaMateAgile/Helpers/PriceHelper.cs:line 54
teslamateagile_1  | 2021-07-04 12:57:40.300 dbug: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Price update complete
teslamateagile_1  | 2021-07-04 12:57:40.301 info: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Waiting 300 seconds until next update
root@teslamate [ /var/lib/teslamate ]#
jrothlis commented 3 years ago

I've updated to rc4, still getting the error.

MattJeanes commented 3 years ago

@jrothlis I was going to say the fix is in RC.3 and above!

Can you please share your TeslaMateAgile logs and configuration please from RC.4? Many thanks

jrothlis commented 3 years ago

Here's the config. The log is as above.

  teslamateagile:
    # image: mattjeanes/teslamateagile:latest
    image: mattjeanes/teslamateagile:v1.7.0-rc.4
    restart: always
    environment:
      - DATABASE_USER=teslamate
      - DATABASE_PASS=supersecretpassword
      - DATABASE_NAME=teslamate
      - DATABASE_HOST=database
      - TeslaMate__UpdateIntervalSeconds=300
      - TeslaMate__GeofenceId=1
      - TeslaMate__Phases=1
      # - TeslaMate__EnergyProvider=Octopus
      # - Octopus__RegionCode=C # Octopus Energy only
      - TeslaMate__EnergyProvider=FixedPrice
      - FixedPrice__TimeZone=Europe/London # IANA (tz database) time zone code, used for below times
      - FixedPrice__Prices__0=00:30-20:30=0.159
      - FixedPrice__Prices__1=20:30-00:30=0.05
      - Logging__Console__FormatterName=simple
      - "Logging__Console__FormatterOptions__TimestampFormat=yyyy-MM-dd HH:mm:ss.fff "
      - Logging__LogLevel__Default=Debug
MattJeanes commented 3 years ago

Thank you @jrothlis - as it turns out there was indeed another bug! I've now re-written the FixedPrice provider to be a lot more simpler and added your failure as a test case, can you please try v1.7.0-rc.5?

paulhargreaves commented 3 years ago

I've nulled all my costs with v1.7.0-rc.5. That has worked, all the values are populated.

e.g. info: TeslaMateAgile.PriceHelper[0] Calculating cost for charges 07/03/2021 23:30:09 UTC - 07/04/2021 03:24:52 UTC info: TeslaMateAgile.PriceHelper[0] Calculated cost 1.28 and energy 28.44 kWh for charging process 8

jrothlis commented 3 years ago

Thank you @jrothlis - as it turns out there was indeed another bug! I've now re-written the FixedPrice provider to be a lot more simpler and added your failure as a test case, can you please try v1.7.0-rc.5?

Success! Thank you for sticking with us!

Attaching to teslamate_teslamateagile_1
teslamateagile_1  | 2021-07-05 13:48:41.577 info: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Price service is starting
teslamateagile_1  | 2021-07-05 13:48:41.581 info: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Using energy provider FixedPrice
teslamateagile_1  | 2021-07-05 13:48:41.582 info: Microsoft.Hosting.Lifetime[0]
teslamateagile_1  |       Application started. Press Ctrl+C to shut down.
teslamateagile_1  | 2021-07-05 13:48:41.583 dbug: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Updating prices
teslamateagile_1  | 2021-07-05 13:48:41.584 info: Microsoft.Hosting.Lifetime[0]
teslamateagile_1  |       Hosting environment: Production
teslamateagile_1  | 2021-07-05 13:48:41.584 info: Microsoft.Hosting.Lifetime[0]
teslamateagile_1  |       Content root path: /app/
teslamateagile_1  | 2021-07-05 13:48:43.033 info: TeslaMateAgile.PriceHelper[0]
teslamateagile_1  |       Calculating cost for charges 06/22/2021 19:38:30 UTC - 06/22/2021 23:34:58 UTC
teslamateagile_1  | 2021-07-05 13:48:43.163 dbug: TeslaMateAgile.PriceHelper[0]
teslamateagile_1  |       Calculated charge cost for 06/22/2021 19:30:00 UTC - 06/22/2021 23:30:00 UTC (unit cost: 0.05, fee per kWh: 0): 1.4428218574305555737425 for 28.85643714861111147485 energy
teslamateagile_1  | 2021-07-05 13:48:43.163 dbug: TeslaMateAgile.PriceHelper[0]
teslamateagile_1  |       Calculated charge cost for 06/22/2021 23:30:00 UTC - 06/23/2021 19:30:00 UTC (unit cost: 0.159, fee per kWh: 0): 0.08418562567833333670272 for 0.52946934388888891008 energy
teslamateagile_1  | 2021-07-05 13:48:43.163 info: TeslaMateAgile.PriceHelper[0]
teslamateagile_1  |       Calculated cost 1.53 and energy 29.39 kWh for charging process 238
teslamateagile_1  | 2021-07-05 13:48:43.264 dbug: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Price update complete
teslamateagile_1  | 2021-07-05 13:48:43.265 info: TeslaMateAgile.PriceService[0]
teslamateagile_1  |       Waiting 300 seconds until next update
MattJeanes commented 3 years ago

Yay! I'll wait a bit more for feedback from others or if anything pops up before releasing v1.7.0 properly but this is good news!

MattJeanes commented 3 years ago

v1.7.0 has now been released fully so this issue will now be closed. Please raise a new issue if you see any further issues with the FixedPrice provider. Thank you!