Integration of large open generation data sets

[charlie9578]

Hi,

I'd be interested in trying to validate P50 estimates by:

1. Predicting the P50 from a short term period (e.g. first 3-years of data)
2. Comparing the predition against future actuals (e.g. the next x-years of data, weather adjusted)
3. Doing this for lots of sites to see if the distribution is centred around the P50, e.g. using EIA (US), ENTSO-E (Europe), Elexon (UK), AEMO (Australia) generation data.

This might be out-of-scope for OpenOA, but it could also allow someone to pick a site from any of these sources, and then predict the P50, which could be very interesting for the industry/market.

Any thoughts?

Thanks, Charlie

[RHammond2]

Hi @charlie9578, thanks again for tracking these ideas here!

I might split this up into two requests:

1. Short-term P50 prediction (based on 2-5 years of data) and validation (compare to remaining x years of weather-adjusted data).
2. Expand openoa/utils/downloader to include EIA, ENTSO-E, Elexon, and AEMO data. This likely means we should have a reanalysis_downloder.py and a generation_downloader.py to separate the reanalysis and generation data access methods.

As for the third point, I'm not sure I entirely followed the purpose of the request. Would you be able to elaborate a bit more on the type of analysis you'd want to see in OpenOA?

[charlie9578]

Hi, yes I think you're right Rob, this is probably best split into two... or three!

The third point is to use the mass of data from lots of sites to see if the predicted P50s from the initial periods, do indeed sit in the middle of the distribution when compared to using a longer period. For instance we could see if the P50 when based on the first few years, is in fact on average higher than what happens over the longer time period, e.g. due to higher system degradation or lower long-term availability.

Perhaps one way to explore this initially is to do something similar with the operational period as for the renanalysis period used, so that we can see if the P50 varies dependent on the number of years included in the operational period , and see if it converges with more years included or drifts in one direction or another (see example of the windiness correction below).

Does that make sense?

Also I like the idea of splitting the downloaders, and I have some initial code from another project for downloading EIA data that can be used as a starting point.

Thanks, Charlie

[RHammond2]

I see what you're saying now, @charlie9578! So my understanding would be that this is more or less a convergence analysis for the number of operational years included in the analysis? I think for a single site this certainly sits in the wheelhouse of OpenOA capabilities, but for lots of sites that might get out of the realm of our current use case to analyze single sites.

I'm not necessarily opposed to the expansion, but I think that could be a good v4 question as to whether we should enable fleet analyses vs the current single farm analyses.

[charlie9578]

I've done a bit of work on this here, which covers both the single wind farm (2 examples below) and the expansion to multiple assets (third image, with 100+ assets using Elexon generation data).

For some assets the result is consistent overtime, but for others there can be significant variations before some convergence. This anlaysis is on the net generation, but I recon things could be improved, particularly for outlying assets, by incorporating availability and curtailment.

Overall though, the central result across all wind farms appears very consistent over time, so would suggest the analysis is solid, and there's no consistent bias, but there's some work needed to reduce the spread.