easystats / parameters

:bar_chart: Computation and processing of models' parameters
https://easystats.github.io/parameters/
GNU General Public License v3.0
438 stars 36 forks source link

Better CIs (and p-values?) for std-coefs #705

Open mattansb opened 2 years ago

mattansb commented 2 years ago

Originally posted on https://github.com/easystats/parameters/issues/655

@bwiernik:

Strictly speaking, both the p values and CIs we currently report for standardized effects aren't strictly accurate because they don't include uncertainty in the means or SDs used to standardize. This is usually ignored by most folks. But, ideally we would include options to compute CIs accounting for those (cf. here and here).

@strengejacke

The supplemental (https://static-content.springer.com/esm/art%3A10.1007%2Fs11336-013-9380-y/MediaObjects/11336_2013_9380_MOESM2_ESM.pdf) has code how to do this, if I understand right. Must take a closer look at it.

bwiernik commented 2 years ago

From the fungible package which implements those methods

https://github.com/cran/fungible/blob/master/R/seBeta.R

https://github.com/cran/fungible/blob/master/R/seBetaCor.R

https://github.com/cran/fungible/blob/master/R/seBetaFixed.R

And somewhat cleaner implementation of the normal-theory methods

https://github.com/psychmeta/psychmeta/blob/b9e27791563f4b98b5e1aaeb5bc97d97bb28d055/R/lm_mat.R#L483

strengejacke commented 1 month ago

This only applies when standardize is not "refit", right?

strengejacke commented 1 month ago

Where do we get all the information, like cov.x or cov.xy, needed for the function arguments?

bwiernik commented 1 month ago

This only applies when standardize is not "refit", right?

Strictly, no, pre-processing the variables still introduces the uncertainty in SD.

but I'm not sure how feasible it would be to wrangle to necessary information in the process to get accurate CIs there.

bwiernik commented 1 month ago

Where do we get all the information, like cov.x or cov.xy, needed for the function arguments?

You can extract the model matrix from the model and compute the necessary covariances using it