SUMM: minimum distance estimation for misspecified models

[josef-pkt]

Suppose we only have "wrong" models, what's the "best" model and our "best" estimates?

And, how do you define "best"?

Motivation: Gamma regression: Greene compares MLE versus GMM, where GMM uses overidentified moment conditions. Gamma regression is in GLM/LEF and uses mean and variance assumptions.

preliminaries

• GLM/LEF can estimate the mean/expected value if only the mean is correctly specified. The estimates are consistent even if these other parts are misspecified. However, what happens if we are not exclusively interested in the mean? What's the "best" Poisson model if our data is overdispersed, the mean function is possibly misspecified and we are also interested in predicting quantiles?
• Misspecified MLE minimizes Kullback-Leibler distance to the true model and parameters converge to pseudotruevalues.
• choosing by optimal prediction accuracy is only a special case with some distance measure for prediction accuracy and mostly used for hyperparameter and model selection (with cross-validation) but usually not for parameter estimation itself. (special case: time series models that minimize squared one step ahead forecast errors for the parameter estimation).

So, What's the point?

• provide and use more general minimum distance estimators
• be clear what the objective function is, i.e. distance measure
• focus on implied final statistic (e.g. average effect, prediction)

... ???

[josef-pkt]

Interesting discussion on information criteria and "all models are false" http://warnercnr.colostate.edu/~anderson/PDF_files/AIC%20Myths%20and%20Misunderstandings.pdf linked to from this stats.stackexchange discussion http://stats.stackexchange.com/questions/86664/can-the-burnham-anderson-book-on-multimodel-inference-be-recommended?rq=1

[josef-pkt]

another reference for information criteria in general settings, including penalized maximum likelihood and robust regression, if the model is misspecified Konishi & Kitagawa (1996), "Generalized information criteria in model selection", Biometrika, 83 http://bemlar.ism.ac.jp/zhuang/Refs/Refs/kitagawa1996biometrika.pdf found in the discussion in http://stats.stackexchange.com/questions/88398/model-selection-with-firth-logistic-regression