Now that we've confirmed that the cfr implementation of GraphEM can run on non-pathological cases, it needs to be upgraded to the next level:
Cross-Validation
The choice of regression model in GraphEM (the graph) is still very unsatisfactory: whether the cutoff radius for a neighborhood graph or the target sparsities of a graphical LASSO ("glasso") graph, the only way to do it now is by trial & error which is unscientific, error-prone, and, frankly, a little embarrassing. We can do a lot better than that with cross-validation.
[x] implement k-fold, block-style cross-validation over the instrumental period (this might entail retooling verif_stats to look at more than the average MSE over the field, for instance). Use the 1-sigma rule. k=5 by default.
[ ] cross-check with pre-instrumental data in pseudoproxy experiments
glasso capabilities
Neighborhood graphs are a quick and dirty way to get a reconstruction, but they underuse the available information. If enough data are available for calibration, glasso can do much better at extracting structure and capturing spatial dependencies. However, glasso is in need of the following updates:
[x] defining reasonable sparsity levels for cross-validation.
[x] allow for hybrid graphs where the climate field graph is obtained through glasso but the proxy-field graph can be neighborhood-based
[ ] compare gains to neighborhood graphs on standard test cases
temperature assumption
as in #2 , this code was written with the assumption that temperature is the only field of interest. Math stays the same, so changing the nomenclature won't change any numerical behavior, but I will still try to:
[x] replace all mentions of "temperature" by "field"
Next level (optional)
[ ] explore the use of skggm, which uses the scikit-learn API to fit GGMs.
[ ] explore the family-wise error rate method to help choosing a sensible graph for the climate part of the covariance matrix, and maybe the climate-proxy part as well. (need Dominique's input)
Now that we've confirmed that the
cfrimplementation of GraphEM can run on non-pathological cases, it needs to be upgraded to the next level:Cross-Validation
The choice of regression model in GraphEM (the graph) is still very unsatisfactory: whether the cutoff radius for a neighborhood graph or the target sparsities of a graphical LASSO ("glasso") graph, the only way to do it now is by trial & error which is unscientific, error-prone, and, frankly, a little embarrassing. We can do a lot better than that with cross-validation.
glasso capabilities
Neighborhood graphs are a quick and dirty way to get a reconstruction, but they underuse the available information. If enough data are available for calibration, glasso can do much better at extracting structure and capturing spatial dependencies. However, glasso is in need of the following updates:
temperature assumption
as in #2 , this code was written with the assumption that temperature is the only field of interest. Math stays the same, so changing the nomenclature won't change any numerical behavior, but I will still try to:
Next level (optional)