Investigate alpha_k = 1 - gamma

[ramhiser]

If effective, demonstrate this and original approaches in paper?

[ramhiser]

After much thought regarding what to do about alpha_k in both the paper and in the code, I've opted to make it a parameter of convenience for flexible modeling.

Paper

Here's a brief blurb that will be added to the paper:

The addition of the parameter $\alpha_k$ is one of convenience and allows the flexibility of various covariance-matrix estimators proposed in the literature. In practice, we generally are not in estimating $\alpha_k$ as it would incorporate $K$ additional tuning parameters to estimate via cross-validation, which is counterproductive to our goal of improving the \emph{RDA} classifier's efficiency.

Code

In the code, we will remove alpha_k as an explicit numerical value passed as a parameter. Instead, we can incorporate some other argument, named shrinkage_family or something similar. The possible values right now are:

• ridge: alpha_k = 1 to resemble ridge regression
• convex: alpha_k = 1 - gamma to resemble Friedman's estimator (need to define the range of gamma's carefully between 0 and 1, inclusively)

More shrinkage_family options could be added later if necessary, but for the HDRDA paper, this approach will greatly simplify things.

[ramhiser]

I suspect that one of the two shrinkage_family arguments will outperform the other. How we determine this should be added to the paper, even if added as supplementary information. The convex argument is possibly more straightforward because the range of candidate values of gamma becomes [0, 1], whereas the values for the standard ridge approach are unbounded. In the latter case, it's not always obvious which maximum value should be considered.

[ramhiser]

I added the shrinkage_type argument to sparsediscrim::hdrda. It has the two options given above. If convex, the range of candidate values of gamma is [0, 1].