vrym2
commented
1 year ago Background: UKPN has a dashboard of GSP-level generation going back a few years for the GSPs that they manage. The generation is of metered sources, and can be downloaded as a CSV file. This data also has less odd behaviour than the UKPN site CSV that we have.
Goal: Determine which GSP has the most solar generation out of the ones on the dashboard. Download the data. Create a data report.
The report should include:
- [ ] Plots of the data
- [ ] The typical day in different times of year (Winter, Fall, Summer, Spring)
- [ ] Compare this GSP solar output with PVLive’s estimate for the same GSP (PVLive’s output should be different as it also estimates non-metered solar generation)
- [ ] Days where the generation was significantly different than normal for that time of year
- [ ] Any missing or odd data that seems to be in the CSV (e.g. some solar data can be negative, etc.)
Second part:
- [ ] Create a NetCDF/Zarr file from the CSV to use in model training. This should be similar to the GSP Zarr.
- [ ] Create an automatic download script to download the most recent data every n hours and add it to the already saved data
@jacobbieker I have looked around the dashboard to download the data into csv, could not find any option to download, am i missing something, do I need any credentials for the access of the data?
jacobbieker
Here is a screenshot of the metered power generation in MW for the last 7 days, I have observed from the raw data that there is a linear rise trend from the 12th hour in mid night of every day in MW value until there is a sudden spike at some timestamp (for example - 2019-12-16 11:00:00) in the morning of the day. I believe this sudden spike is the actual true power generated from the panels, and the linear rise before that is just dummy data, and this trend is followed in every other day of the dataset. We can determine the trend change by looking at the slope
peterdudfield
Data Link: https://dsodashboard.ukpowernetworks.co.uk/d/jnTAkg7Gz/ukpn-dso-dashboard-grid-supply-areas?orgId=1&from=now-1y&to=now&refresh=10m&var-GSP=CANTERBURY%20NORTH
Background: UKPN has a dashboard of GSP-level generation going back a few years for the GSPs that they manage. The generation is of metered sources, and can be downloaded as a CSV file. This data also has less odd behaviour than the UKPN site CSV that we have.
Goal: Determine which GSP has the most solar generation out of the ones on the dashboard. Download the data. Create a data report.
The report should include:
[x] Plots of the data
[x] The typical day in different times of year (Winter, Fall, Summer, Spring)
[x] Compare this GSP solar output with PVLive’s estimate for the same GSP (PVLive’s output should be different as it also estimates non-metered solar generation)
[x] Days where the generation was significantly different than normal for that time of year
[x] Any missing or odd data that seems to be in the CSV (e.g. some solar data can be negative, etc.)
[x] All datetimes are British Summer Time. Change to to UTC using the process on the napkin (Dan suggested to do this as well!)
Second part:
Bonus: