py-econometrics / pyfixest

Fast High-Dimensional Fixed Effects Regression in Python following fixest-syntax
https://py-econometrics.github.io/pyfixest/
MIT License
176 stars 35 forks source link

Case Study: Multiple Testing Corrections / Family Wise Error Rates #676

Open s3alfisc opened 3 weeks ago

s3alfisc commented 3 weeks ago

Would be cool to add a case study that introduces to family wise error rates and how to handle them via pf.bonferroni(), pf.rwolf() or simultaneous confidence bands.

This could be done by simulating a dgp with multiple dependent variables under true nulls (i.e. no treatment effect) and by counting false positives under standard inference, Bonferroni & Romano-Wolf corrections.

In a second step, we could then introduce the concept of "uniformely more powerful" tests by comparing the power of the Bonferroni vs RW method of detecting a true effect.