search

worldbank / ietoolkit

Stata commands designed for Impact Evaluations in particular, but also data work in general
https://worldbank.github.io/ietoolkit/
MIT License
214 stars 74 forks source link

[iebaltab]: correct [stdev] option to re-calculate standard deviations #243

Open mizuhirosuzuki opened 4 years ago

mizuhirosuzuki commented 4 years ago

Sometimes I want to show standard deviations of variables in each group, not standard errors. It may be nice to have an option to show SDs rather than SEs. When choosing to use SDs, options such as vce and control are just redundant, so you may want to show error messages when SD option and other unnecessary options are both specified in the same command.

mizuhirosuzuki commented 4 years ago

Sorry, I just found this feature...

kbjarkefur commented 4 years ago

Could it have been made more clear in any way?

mizuhirosuzuki commented 4 years ago

I wanted exactly what "stdev" option does. But might be useful to give error messages when "stdev" and "vce" options are used simultaneously. In my understanding, "vce" option is used to calculate standard errors in some ways (eg. clustering at the village level), so specifying the "vce" option should be meaningless when "stdev" option is used. Standard deviations are just a dispersion in the data, no matter how they are sampled.

kbjarkefur commented 4 years ago

I trust you more than myself when it comes to what is expected or standard in research. @luizaandrade , what do you say? @bbdaniels maybe you can add something as well.

bbdaniels commented 4 years ago

@kbjarkefur @mizuhirosuzuki is correct, in that the functionality needs to be fixed in Line 1589. Reverse-engineering the SD from the regression SE will be incorrect if vce() is applied. This line should re-calculate the SD from data and potentially also remind the user that the SD is an exact measure of the data and not adjusted in any way (whereas the SE is an estimate).

https://github.com/worldbank/ietoolkit/blob/ea8f1e2d308d4b99c28f0d178c716b9cdb2002b9/src/ado_files/iebaltab.ado#L1589