SolarArbiter / solarforecastarbiter-core

Core data gathering, validation, processing, and reporting package for the Solar Forecast Arbiter
https://solarforecastarbiter-core.readthedocs.io
MIT License
33 stars 21 forks source link

refactor reference forecasts in metrics #386

Closed wholmgren closed 4 years ago

wholmgren commented 4 years ago

https://github.com/SolarArbiter/solarforecastarbiter_dashboard/issues/214 discussed the need to implement reference forecast metrics on a per observation/forecast basis rather than a single reference forecast for all observation/forecasts. Most relevant part is quoted below:

Looking again at the core calculator and preprocessing code, I'm a little concerned that we might need to change things so that the reference forecast is validated along with the forecast and observations. This would make it easier to supply a unique reference forecast for each forecast/observation pair in core, which could be used or not in the dashboard/api.

I can imagine that people might want to do something like "compare forecast skill metrics for each test forecast at each SURFRAD site". You end up with a graph that shows forecast skill like any other metric. Perhaps the labels are twice as long for clarity about what's being shown.

I had forgotten that the data model already supports one reference forecast per fx/obs pair:

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/7c933afe97307a35057a9eeae1bc92e60d129d7b/solarforecastarbiter/datamodel.py#L922

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/7c933afe97307a35057a9eeae1bc92e60d129d7b/solarforecastarbiter/datamodel.py#L951

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/7c933afe97307a35057a9eeae1bc92e60d129d7b/solarforecastarbiter/datamodel.py#L1156

It appears we need to do the following in core:

https://github.com/SolarArbiter/solarforecastarbiter_dashboard/pull/220 implements the dashboard front end for this, including a new reference_forecast key under object_pairs in the report's json specification.

alorenzo175 commented 4 years ago

Add reference forecast information to reports/templates/metrics_meta_table.html, obsfx_table.html, or a new table. If we put it in the metrics_meta_table, the new column would probably just contain the name (with link) or None. The obsfx_table would make sense to be clear about the resampling/alignment (or lack thereof). But I don't know if we can cram more into that table.

Do we really need the interval label/length for the observation and forecast in the obsfx_table when we provide the links to the metadata? It would free up space for a reference forecast column. Alternatively move the interval information to another table