Open seabbs opened 1 year ago
There is also this point from the discussion which covers many of our use cases:
Of course, stronger methods might become available if structure can be exploited; for example, in sequential time-series forecasts for one step ahead, in addition to the predictive quantiles being calibrated, exceedances ought to be independent (Christoffersen 1998).
nice! Still have to read the paper though. Maybe @jbracher and other friends of the group would even be interested in contributing code?
The paper has code attached which seems like a pretty good starting point for a conversation!
@jbracher penny for your thoughts!?
A new paper from @jbracher and friends: Model Diagnostics and Forecast Evaluation for Quantiles (http://dx.doi.org/10.1146/annurev-statistics-032921-020240)
I found this a little hard to read if I am being honest (i.e. it took me a while) so when implementing any of the ideas will need to provide some nice docs vs just relying on pointing at this work.
It has some really nice ideas in it for evaluating calibration and plotting discrimination and miscalibration that we could think about supporting. I assume the authors would be happy with us doing this (@jbracher)?
I particularly like these plots.