search

mpiktas / midasr

R package for mixed frequency time series data analysis.
http://mpiktas.github.io/midasr/
Other
74 stars 34 forks source link

Average Forecast Combination Weight Assignment #90

Closed bill-spec closed 1 year ago

bill-spec commented 1 year ago

Hi,

This package is great.

I have a question regarding the implementation of the forecast combination weights used in the average_forecast function. When combing forecasts, I always understood that a forecast's assigned weight should be time varying. Since, for example, if a model had a great performance in t+4, I would not want to use that information to inform the assigned weight it would get to forecast t+1.

It seems that the average_forecast function does not do this. As I read it, after all the recursive modeling is done the out of sample forecasts just get their weights calculated by the entire sample. Am I missing something (reading the code incorrectly, I did some live testing to confirm my thoughts before asking this too).

With all of that being said, assuming I understood this implementation correctly, can someone explain why this was the chosen approach to forecast weighting? I could not find any mention of this in the JSS paper. I especially don't understand exactly how the discounted MSFE approach is supposed to be used here.

Thanks!

vzemlys commented 1 year ago

Forecast weights in the package are fixed. The choice between time varying and fixed costs for the particular problem should be based on the data. In general I assume that it would be quite difficult to prove that varying weights are beneficial, as then you should make some assumptions how they vary. The code for weighted forecasts was in its idea illustrative, it should not be hard to modify it to varying weights in a recursive setting. Do you have any research which shows that this is technique brings benefits?

bill-spec commented 1 year ago

Thanks for the fast response! I do have a working modified version accomplishing what I wanted. What I really wanted to hear about was your though process and confirm I was understanding the code correctly which your comment answers perfectly.

The idea for time varying weights is coming from the 2013 paper: Should Macroeconomic Forecasters Use Daily Financial Data and How? Maybe the weights don't have to vary at every period I am novice when it comes to this literature.

I do think however the current code calculating weights using out of sample data, then returning a weighted forecast on the same period seems like look ahead bias but I have to come back with more concrete examples of what I mean.

Thanks again for the response!