search

mike-lawrence / ez

Development repository for the `ez` package for R
52 stars 17 forks source link

Incorrect between_covariate result. #60

Open slfan2013 opened 6 years ago

slfan2013 commented 6 years ago

The between_covariate result is wrong. We cannot simply take the residual of lm(dv~between_covariate) and regress on other between variables.

For example, 

res = lm(dependent_variable~covariate)
lm(res~independent_variable)

is not equivelent to 

lm(res~independent_variable + covariate)

The first one will produce a high risk of false discovery.

mike-lawrence commented 6 years ago

Hm, in my simulations it's the latter that yields a high rate of false alarms, and only in the case where the covariate is correlated with the independent variable, where you're not supposed to be doing regression like that in the first place.

slfan2013 commented 6 years ago

Please have a look at https://stats.stackexchange.com/questions/286850/linear-regression-confounder

mike-lawrence commented 6 years ago

Please have a look at my simulation here, which produces this plot. Feel free to correct any code or interpretation error I've made.

slfan2013 commented 6 years ago

No, I am not talking about the high or low risk of type I error. I am saying that the statistic of dealing with covariate in ez package is incorrect. Please see this discussion.

mike-lawrence commented 6 years ago

That discussion highlights that the method I use is only a problem when the covariate is correlated with one or more predictors, in which case yes, you should be doing multiple regression. I think the problem here is that people use different terminology for these things. ANCOVA was originally intended as merely a means to increase power for the analysis of the relationship between Y~X when there exist measurements for Z which has a known effect on Y but is not correlated with X and does not interact with the effect that X has on Y. When these conditions hold, subtracting the variance associated with the Z effect on Y will improve power to assess the effect that X has on Y. The case where Z & X are correlated or interact is clearly a matter for multiple regression, and the term "ANCOVA" doesn't really make any sense. I will add a warning that appears when users supply covariates to ezANOVA notifying them of the abova assumptions.

-- Mike Lawrence Graduate Student Department of Psychology & Neuroscience Dalhousie University

~ Certainty is (possibly) folly ~

On Tue, May 29, 2018 at 12:54 AM, slfan2013 notifications@github.com wrote:

No, I am not talking about the high or low risk of type I error. I am saying that the statistic of dealing with covariate in ez package is incorrect. Please see this discussion https://stats.stackexchange.com/questions/286850/linear-regression-confounder .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mike-lawrence/ez/issues/60#issuecomment-392645645, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJM_UfzQS0xmJEJ_rMyG-8xc-0RabCBks5t3MZhgaJpZM4ULSM9 .