Record optimal tuning parameters from classification study

[ramhiser]

Summarize in paper. How?

Possibilities:

• Table
• 2D Heatmap

[ramhiser]

Today, @steincaleb and I discussed how we can report the optimal values. Once we have results, we can explore the usage of a 2D heat map. Some scenarios to consider:

1. Display heat maps of optimal values for each data set for a fixed value of d
2. Display heat map for one data set and a single value of d

Of course, we will choose the heat maps for the cases where we perform the best. However, the heat maps that we choose will ideally reflect the relaxation of the linearity assumptions employed in the LDA classifier. Given that the majority of classifiers designed for p >> n are variants on LDA, the heat maps should support the claim that the linear assumptions are too strong.

[ramhiser]

Although we said heat maps before, I found them confusing due to the sparsity of the majority of values as we will see below. Instead, I have created quick-and-dirty stacked bar graphs. I really don't like stacked bar graphs, especially for a publication, but they tell an important story for our preliminary analysis. It's very interesting to see the bathtub shape in terms of the optimal values of lambda for the majority of the data sets.

Recall that for each data set, there are 1000 replications for each number of variables selected d. The data sets are facetted by d, which are labeled at the top of each subfigure.

Burczynski Data Set

Nakayama Data Set

Shipp Data Set

Singh Data Set