MetPX / sarracenia

https://MetPX.github.io/sarracenia
GNU General Public License v2.0
45 stars 22 forks source link

adding wiskis query support #758

Closed petersilva closed 1 year ago

petersilva commented 1 year ago

This is a copy of an internal to Government of Canada request... there is more detail there... Briefly:

request from client:

We would like your assistance in acquiring data from the Ontario Power Energy. They changed their protocol of transfer to the new WISKI system. We would like to do tests with a sample of data before to switch to the new system.

This is a reference to: https://www.kisters.net/wiski/

and there are some python bindings here: https://pypi.org/project/kiwis-pie/

and we also have a VB sample from OPG (omitted because it mixes credentials and code.)

A substantial complication is that one uses a username to obtain a token, and then do the queries using the tokens for authentication.

@Rehouma63 provided an initial proof of concept standalone python script that obtained a sample time series. I have re-worked it to fit into sarracenia for eventual deployment.

petersilva commented 1 year ago

@Rehouma63 finished his term, so I picked up where he had gotten to and brought it a little further. Here is a sample file being produced:

fractal% more /tmp/wiski/ts_23755_2023-09-19h01:19:44.629537+00:00_2023-09-20h01:19:44.629537+00:00.csv Timestamp,Value 2023-09-19 00:00:00-05:00,390.07 2023-09-19 00:01:00-05:00,390.08 2023-09-19 00:02:00-05:00,390.1 2023-09-19 00:03:00-05:00,390.1 2023-09-19 00:04:00-05:00,390.11 ... (time series covers 24 hours and has a single value in it: Air Temperature)

things that puzzle me:

petersilva commented 1 year ago

progress:

petersilva commented 1 year ago

ok latest commit, I get some data, and write a bunch of timeseries files.

petersilva commented 1 year ago

sample released with 3.0.44