Open benthestatistician opened 9 years ago
Separable from but related to broader proposal described/discussed in this wiki
Noting a tentative resolution (from offline discussion with @markmfredrickson) to fold this in with current developments on the clusters branch.
It's a little odd that balT()
's reports both an overall p-value and univariate p-values, without coordination between the two. Jotting down notes on 2 potential ways the one could inform the other.
Re 1, I have a conjecture: The multivariate test, giving p-value q0
, can be combined with univariate z-tests giving p-values q1
, ..., qk
to give a Holm-like, FWER-controlling, step-down procedure furnishing p-values p1
, ..., pk
that are uniformly no larger than p.adjust(c(q1, <...>, qk), method="holm")
.
If the conjecture turns correct, we might consider offering that procedure as an alternative to the other p.adjust options. Of course verifying the conjecture may also suggest elaborations giving still better power.
Re 2, one might follow up on a significant global test by reporting the significance of balance tests performed on principal components of the x-matrix, rather than on the x-es themselves. These correspond more closely to the global test statistic, so this approach could facilitate coherence among the tests.
The univariate p-values that we are currently providing are not corrected for multiplicity, something we should correct. Let's do this by passing the current univariate p-values through
stats:p.adjust
:xBalance
argumentp.adjust.method
(to be passed down asmethod
arg top.adjust
).p.adjust.method
defaults to "holm".p.adjust.method
suggests "BH", "fdr", "BY", and "none", as other possible values, also indicating that "none" gives the <= 0.1-12 behavior.xBalance
'sp.adjust.method
argument is passed through tostats:p.adjust()
. Also states that "hochberg" or "hommel" adjustment methods require dependence assumptions that need not be met in covariate imbalance calculations.