openclimatefix / ocf-ml-metrics

Collection of simple baseline models and metrics for standardized evaluation of OCF forecasting models
MIT License
4 stars 0 forks source link

Financial error metric #4

Open dantravers opened 1 year ago

dantravers commented 1 year ago

Add new metrics to capture the financial impact of the improved PV forecast. This isn't what is called a "proper scoring metric", in that a worse forecast can actually help your financial metric if you happen to be lucky. But its actually very valuable to be able to quantify the financial impact of our forecasts for end users.

The financial loss metric is relevant for day ahead forecasts (as it is comparing the day ahead price with the balancing price). It could in theory be applied to forecats a few hours ahead, but it's relevance is primarily for day-ahead forecasts. It can be used for site-level forecasts, but also for national forecasts.

The theory is in the screenshot attached: image

The figures s_a and s_f are the target / forecast. The figures p_f, p_rt are the EPEX and Balancing (SSP / SBP) prices in the csv attached.

The figures are calcualted for each hour (take sum of forecast and actual generation figures across 2 half-hours to convert to hours), and then the expectation of the hourly figures calcualted across the period being tested - this coul be a year, by month, or by time of day, etc.

dantravers commented 1 year ago

epex_prices.csv ssphourly.csv