search

SolarArbiter / solarforecastarbiter-core

Core data gathering, validation, processing, and reporting package for the Solar Forecast Arbiter
https://solarforecastarbiter-core.readthedocs.io
MIT License
34 stars 21 forks source link

Add reference event observations and forecasts #429

Open wholmgren opened 4 years ago

wholmgren commented 4 years ago

The point is just to demonstrate functionality. Maybe something like AC power ramps.

dplarson commented 4 years ago

What's the overall workflow for adding reference datasets? Create the CSV file (formatted according to https://solarforecastarbiter.org/datamodel/#uploads) and attach it to the relevant open issue?

lboeman commented 4 years ago

What's the overall workflow for adding reference datasets? Create the CSV file (formatted according to https://solarforecastarbiter.org/datamodel/#uploads) and attach it to the relevant open issue?

For all the existing reference data we have fetching code spread between pvlib and https://github.com/SolarArbiter/solarforecastarbiter-core/tree/master/solarforecastarbiter/io/fetch

We also have the reference_observation module that has code to initialize sites and observations, and functions to coordinate the fetching procedure. https://github.com/SolarArbiter/solarforecastarbiter-core/tree/master/solarforecastarbiter/io/reference_observations

There's a csv of reference site metadata for initializing sites (https://github.com/SolarArbiter/solarforecastarbiter-core/blob/master/solarforecastarbiter/io/reference_observations/sfa_reference_sites.csv) and a few source-specific sub-modules (e.g. srml.py) implementing a number of functions to initialize observations at each site, which are utilized by common.py.

Data is uploaded to the framework using the referencedata command of the cli (which utilizes: https://github.com/SolarArbiter/solarforecastarbiter-core/blob/master/solarforecastarbiter/cli.py The fetching and uploading is carried out by a cronjob, which @alorenzo175 has configured.

Hopefully that gives you an idea of all the moving parts.

lboeman commented 4 years ago

Woops, solarforecastarbiter.io.reference_observations.reference_data is the module that coordinates all the initialization, not common.py.

wholmgren commented 4 years ago

@dplarson we figured that there would be so little user-provided public reference data that it was not worth implementing a front end for it.

@lboeman that's a great description. We should put it somewhere where someone can find it. Perhaps a new top-level page in the core documentation.

One kind of reference event forecast that would be easy to implement is GHI clear/not clear. The forecast would be made by comparing the forecast GHI to a clear sky model. We'd make new observations that are the result of the data validation toolkit / pvlib's clear sky detection algorithm.

dplarson commented 4 years ago

@dplarson we figured that there would be so little user-provided public reference data that it was not worth implementing a front end for it.

That makes sense to me. And my question was meant solely to help me understand the process for adding other reference datasets, e.g., net load observations (issue #431).

Also, I think your suggestions of AC power ramps and GHI clear/not clear are both good.