search

robjhyndman / forecast

Forecasting Functions for Time Series and Linear Models
http://pkg.robjhyndman.com/forecast
1.1k stars 340 forks source link

Robust long run variance estimation for DM-Test #898

Closed KuKi2121 closed 1 year ago

KuKi2121 commented 2 years ago

Formal Motivation: Estimating the long-run-variance of the residual differences by means of the autocorrelation function could lead to negative estimates for a prediction horizon above one (h >1) and therefore for a not calculatable test. Harvey, Leybourne and Whitehouse (2017)

Status Quo: If this appears to happen, the implemented DM-Test of the forecast package prints a warning and forces the prediction horizon to one, and does the test all over again for h=1.

Two possible methodological Handlings:

  1. The use of bartlett weights could be introduced. Suggested in Harvey, Leybourne and Whitehouse (2017) as well as in Diebold and Mariano (1995).

  2. Calculating the DM-Statistic by regression of the loss differential on an intercept on an intercept, using heteroscedasticity and autocorrelation robust standard errors. So one could easily use the Newey-West-Estimator that ensures a positive long-run-variance estimation. This was suggested by the tests founding father in Diebold (2015). A possible implementation could look like the following: ` d <- abs(e1)^power-abs(e2)^power

    n <- length(d) D <- data.frame(lossdiff=d) R <- lm(lossdiff~1,data=D) mean_d <- R[["coefficients"]] varest <- sandwich::NeweyWest(R,lag=h-1,prewhite=FALSE) mdm <- sqrt((n+1-2h+h(h-1)/n)/n) STATISTICS <- mdm*mean_d/sqrt(varest) `

I would be really happy to help by contributing an enhanced version of the existing dm.test function.

Best & Thanks Kirill

robjhyndman commented 2 years ago

Happy to consider a PR provided the resulting function can still be considered a DM test. I'd also like to avoid additional package dependencies if possible.