wbnicholson / BigVAR

Dimension Reduction Methods for Multivariate Time Series
56 stars 17 forks source link

Lamda grid parameters question. #36

Open CopulaCoVaR opened 2 years ago

CopulaCoVaR commented 2 years ago

Hi Dr. Nicholson, I'm trying to understand the vector of parameters of the gran() argument in the constructModel function. Your BigVAR guide states, "The first option controls the depth of the lambda grid (a good default option is 50). The second option controls the number of grid values (a good default is 10)."

  1. My question refers to what is meant by "depth of the Lambda grid" .
  2. On the other hand, it is unclear how to choose the second option in gran().

I would really appreciate your help or guidance in finding resources that can help me solve this inquiry.

wbnicholson commented 2 years ago

The lambda grid starts by finding the theoretical upper bound representing the smallest value in which all coefficients are set equal to zero which is then refined by a binary search to find a smaller empirical value.

The grid of candidate values then decrements from this value in either log-linear (default) or linear increments before truncating at the maximum value divided by the first parameter of gran. Larger values of gran create in a deeper grid which results in less sparse coefficient matrices.

The second parameter in gran sets the coarseness of the penalty grid creating gran[2] potential candidate lambda values for use in rolling validation. Setting this to a very large number may allow for more precision but will substantially increase the computational burden of the rolling validation procedure. I've generally found that there is little marginal forecasting improvement beyond 10 candidate parameters.

CopulaCoVaR commented 2 years ago

Thank you! I was able to manually run the algorithm for the Lambda grid and it worked perfectly!

I'm currently working on a function that helps me select the best gran[1,2] for a given dataset in order to get a better approximation of the optimal lambda regardless of the computational burden.

In doing this two related questions emerged:

1- Which one of the following cv.BigVAR output statistics would be the appropriate criteria to select the best model?

I tried with mean(OOSMSFE) given that the OOSMSFE itself is a vector and not a single numerical value. Is this the right approach?

2- How is the seoosfmsfe defined? Is it the standard error of the OOSMSFE? On the other hand, does the smallest MSFE always returns the smallest seoosfmsfe?

Lastly, I would like to ask you where I can find the latest version of "Structured Regularization for Large Vector Autoregression" as I found some minor typos on the version I'm working with (September 25, 2014).

Thank you beforehand. I'm really amazed by the work you've done with this package, that's why I'm trying to understand and apply it fully!

wbnicholson commented 2 years ago

I wouldn't bother trying to optimize the second parameter of gran. Since it just sets the number of candidate values, a larger number will generally improve performance with diminishing marginal returns.

If you want to optimize over the first parameter, I'd recommend looking at InSampMSFE which contains the in-sample MSFE of each potential candidate lambda value. If the last column has the smallest MSFE that indicates that the penalty grid might not be deep enough. I included an example in the user guide.

How is the seoosfmsfe defined? Is it the standard error of the OOSMSFE?

Yes, it's the standard error of MSFE over the forecast evaluation period. It's not necessarily the case that the in-sample minimize of MSFE will have the smallest standard error; inducing more regularization by using the "one standard error heuristic" (setting ONESE=TRUE in constructModel) may have smaller standard error.

Lastly, I would like to ask you where I can find the latest version of "Structured Regularization for Large Vector Autoregression" as I found some minor typos on the version I'm working with (September 25, 2014).

The arxiv preprint is the latest version of the paper.