DM test with score instead of forecasts

jeslago / epftoolbox

An open-access benchmark and toolbox for electricity price forecasting

Apache License 2.0

190 stars 73 forks source link

DM test with score instead of forecasts #27

Open simon-hirsch opened 2 weeks ago

simon-hirsch commented 2 weeks ago

Hi,

would you accept a PR for a DM-test version that accepts scores_1, scores_2 as inputs instead of the original prices and point forecasts? That way, the DM test could as well be used for probabilistic forecasts if one passes two (n, ) or (n, d) arrays of CRPS (or Energy Scores, Variogram Scores, ...). Currently the scope is somewhat limited to the point forecasting case.

Cheers, Simon

enertel commented 2 weeks ago

Check out https://github.com/ogrnz/feval - you can pass either point or probabilistic loss scores and it will do a more general case of the DM test, it even has built in procedures for identifying model confidence sets. Highly recommend :)