easystats / parameters

:bar_chart: Computation and processing of models' parameters
https://easystats.github.io/parameters/
GNU General Public License v3.0
440 stars 36 forks source link

Should `model_parameters(<a mixed model>)` use `ci_method = "S"/"KR"` by default? #989

Open mattansb opened 4 months ago

mattansb commented 4 months ago

The current default for mixed models is to use residual dfs, but these are counter conservative. Should we default (when possible) to use Satterthwaite or Kenward-Roger dfs instead?

strengejacke commented 4 months ago

I'm not sure if this should be the default. Maybe Sattherthwaite, but K-R can be really slow on large data sets.

Unlike many (or every?) other software, model_parameters() exactly describe the approximation methods, so we're already very transparent, which kind of DF are used to compute the p-values, so I would also be fine when we change nothing. But I'm open to switching to SW (not K-R though).

bwiernik commented 4 months ago

The performance of the various approximations varies so much by model type and features (eg number of clusters, cluster size heterogeneity) that it's not clear to me which if any is most reasonable to choose as a default.

Cf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4458010/ vs https://link.springer.com/article/10.3758/s13428-016-0809-y

Perhaps changing to S df for gaussian models but sticking with residual or between-within for other families?

mattansb commented 4 months ago

Do have any support for dfs for none gaussian models?

Anyway, I support this:

Perhaps changing to S df for gaussian models

strengejacke commented 4 months ago

We have as approximation methods