florianhartig / DHARMa

Diagnostics for HierArchical Regession Models
http://florianhartig.github.io/DHARMa/
208 stars 22 forks source link

Diagnosing residuals of custom variance estimators from "sandwich" #347

Closed DexterGlass closed 2 years ago

DexterGlass commented 2 years ago

The "parameters" R package allows one to try Heteroskedasticity and Autocorrelation consistent estimators, HC estimators suitable for small sample sizes via paired bootstrap, but there's no current support in dHARMA for diagnosing whether those alternative variance estimators actually fixed the heteroskedasticity and estimation problems.

TLDR: The current dHARMA package doesn't support checking mixed models with custom variance estimators.

parameters documentation: https://easystats.github.io/parameters/articles/model_parameters_robust.html

florianhartig commented 2 years ago

As for #339, I don't see how this type of models can be tested with the principles of DHARMa. Again, the idea of of these estimators is not that they generate an explicit statistical model of heteroskedasticity (which is done, e.g., in glmmTMB, and can be tested), but only that they are ROBUST to it (so the problem is still there, but the way the parameters are estimated is different and hopefully not affected by the problem).

What you can do with DHARMa is to test the base models of your example (e.g. lm, lme4). Once you detect a problem, and you choose a robust estimator, you basically have to "trust" that the estimate is now robust to the problem, but I can't test if that is the case, because the problem is still there.