search

SolarArbiter / solarforecastarbiter-core

Core data gathering, validation, processing, and reporting package for the Solar Forecast Arbiter
https://solarforecastarbiter-core.readthedocs.io
MIT License
33 stars 21 forks source link

clear sky normalization in metrics/reports #226

Open wholmgren opened 4 years ago

wholmgren commented 4 years ago

I'd like to consider the possibility of computing metrics for on-the-fly calculations of clear sky index observations and forecasts. So, user selects a GHI forecast, a GHI observation, and indicates that they would like metrics for both the GHI and the framework-calculated CSI. Or for PV power using the provided plant metadata.

I think we've discussed this in the past but I don't actually see this in the use cases document. The only similar thing I see in the use cases document is normalization by nameplate power.

We already have the functions to do this. The real challenge is probably keeping track of yet another permutation in reports interface, metrics calculation, and templated report.

What do you guys think @awig @dplarson @alorenzo175

dplarson commented 4 years ago

I'm in support of this in general, although I agree there are some code design issues to work through.

At the same time, figuring this out may also lead to improvements in the user experience (e.g. go towards dynamic filtering in the report interface where a user can select a window of data and compare the total error metrics against the error metrics for the selected window). Not saying that's necessarily what we should do, but just mentioning that this may bring up new ideas/designs (and solutions).

wholmgren commented 4 years ago

By "on-the-fly" I meant "calculated when the report is run" rather than a dynamic metrics calculation interface within the already generated report. The dynamic interface is currently not possible due to the security model of the dashboard/API, but I agree it would be cool in the long run.

dplarson commented 4 years ago

Gotcha. Thanks for clarifying.

wholmgren commented 4 years ago

This issue looks more doable now that #370/#379 added support for normalization for power. We had some discussion about normalized metrics for irradiance and decided that it did not make sense to support them at the time.

A key like "normalization": "clearsky_values" could indicate that normalized metrics should be for clear sky index, then the preprocessing functions would put the right clearsky data in ProcessedForecastObservation.normalization_factor.

At the risk of getting off topic... Maybe just "clearsky" would work, but I was also thinking that keys like "clearsky_daily", "clearsky_monthly", or "clearsky_annual" could be used to indicate normalization by time averaged values (and there would be similar for climatology but that's more complicated since it could require another data fetch).