search

stevenewey / octograph

Python tool for extracting Octopus Energy meter readings to InfluxDB
98 stars 48 forks source link

Enhancement request. Agile future rates #6

Open Womble-1 opened 5 years ago

Womble-1 commented 5 years ago

It seems that the Agile rates are limited to the past. It would be good if the script also populated the published future rates.

-thanks for developing and sharing!

WiteWulf commented 4 years ago

I've just moved to Agile and came here to comment on the same thing. If I specify a '--to-date' of a day in the future I note that it grabs the agile rates for the forthcoming day, but grafana doesn't plot them, even if I again specify a date range extending a day into the future.

Either influxdb isn't logging the data from the script or grafana isn't plotting it on the chart.

Womble-1 commented 4 years ago

I ended up pulling the rates in via telegraf into a new Measurement.

Grafana will show future points, either with something like to: "now+30h" or if you want to go to a fixed time of day in the example tomorrow at 8am this works "now/d-1d+32h" . There may be better (proper?) ways to do this, but these work.

telegraf.conf snippet

[[inputs.http]] interval = "3600s" urls = [ "https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-H/standard-unit-rates/" ] method = "GET" data_format = "json" name_override = "agilerates" json_query = "results" json_time_format = "2006-01-02T15:04:05Z" json_timezone = "UTC" json_time_key = "valid_to"

garyp-work commented 4 years ago

Yeah, I just had a poke around in the influx database and can confirm that, although the python script is pulling the future agile rates in if you specify a --to-date in the future, the values aren't getting written into the database.

psaffrey commented 3 years ago

I can see why this is happening - the loops that build the payloads to send to InfluxDB work on the metrics and augment these with agile data, so they won't upload points if they only contain agile data.

I've managed to get it to store future data as well, although it requires some other changes since the tag structure for the entries that are uploaded currently depend on the Go tariff high/low period. I don't think the tag really needs this and removing it allows me to upload both types of points - those in the past where we have measurements and those in the future where we only have rates. I still need to test whether the in-the-future points are properly updated when they're not in the future anymore.

If anybody has an interest in these changes I can upload a fork or make a PR to this repo...?

Womble-1 commented 3 years ago

I would make use of this if you made the change. As long as it doesn't break the referencing of the data that is already stored. Thanks...

psaffrey commented 3 years ago

I ended up with my own fork:

https://github.com/psaffrey/octograph

It looks like there are already quite a few forks of this repo. There are many like it but this one is mine.

This includes the feature we're talking about above, but also includes the ability to not ask for gas (by omitting that section from .ini file) and a "dry-run" mode to show what would be uploaded to InfluxDB without actually uploading it (useful for testing and debugging).