search

forc-db / Global_Productivity

Creative Commons Attribution 4.0 International
2 stars 0 forks source link

testing for interactions #61

Closed beckybanbury closed 4 years ago

beckybanbury commented 4 years ago

@teixeirak

I learned today that it apparently it isn't good practice to use linear mixed models to test for interactions between two continuous variables, which would affect the interaction tests we've done. Do you (or @ValentineHerr ?) know anything about this/have any thoughts? Apparently a better option is to use GAMs - I think I could fairly quickly redo the MATxMAP interaction tests if needed using GAMs, but I'm not entirely sure whether or not this is actually necessary!

ValentineHerr commented 4 years ago

I have never heard of this. Do you have a reference I could look at? Are you talking about interactions in the fixed effects or random effects?

beckybanbury commented 4 years ago

I haven't managed to find any information on it online - it came up in conversation with a couple of people in the department. I'm looking at interactions in the fixed effects - as I understand it the problem seems to be specifically that they're both continuous variables and a linear model won't account for changes in the shape of the relationship, only in the slope. I'm not convinced that this matters particularly for our purposes, but thought it was worthwhile to check!

teixeirak commented 4 years ago

Makes sense, and wouldn't hurt to try if you feel inspired, but I agree that its unlikely that it would alter our conclusions. I'd be fine leaving as is.

beckybanbury commented 4 years ago

I'm quite happy to not redo it, especially as we don't have enough data to fit a GAM to R_auto and R_auto_root. Given that we aren't trying to draw conclusions about the specific shape of the relationship, only the fact that there is an interaction, it seems that it shouldn't matter too much.

teixeirak commented 4 years ago

Okay, closing this.

beckybanbury commented 4 years ago

Sorry - just to come back to this: in trying to present a table of the results of the interaction tests, I've found that for some of the fluxes I get a different best model selection depending on whether I use the p-value or the AIC value. E.g. for GPP, AIC selects a model with an interaction effect, whereas an anova selects a model with an additive effect. This is because the anova doesn't identify a significant difference between the additive and interactive model, despite there being a significant difference between the 'null' and additive model (see outputs below).

I think the conclusion would be that there is a significant additive and interactive effect, but no significant difference between the two models, but how should we present this?

Data: df Models: mod.single: mean ~ mat + (1 | geographic.area/plot.name) mod.add: mean ~ mat + map + (1 | geographic.area/plot.name) mod.int: mean ~ mat * map + (1 | geographic.area/plot.name) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
mod.single 5 1206.4 1223.9 -598.21 1196.4
mod.add 6 1192.9 1213.9 -590.45 1180.9 15.5045 1 8.231e-05 *** mod.int 7 1192.7 1217.1 -589.34 1178.7 2.2211 1 0.1361

Signif. codes: 0 ‘’ 0.001 ‘’ 0.01 ‘’ 0.05 ‘.’ 0.1 ‘ ’ 1

anova(mod.single, mod.int) Data: df Models: mod.single: mean ~ mat + (1 | geographic.area/plot.name) mod.int: mean ~ mat * map + (1 | geographic.area/plot.name) Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
mod.single 5 1206.4 1223.9 -598.21 1196.4
mod.int 7 1192.7 1217.1 -589.34 1178.7 17.726 2 0.0001416 ***

Signif. codes: 0 ‘’ 0.001 ‘’ 0.01 ‘’ 0.05 ‘.’ 0.1 ‘ ’ 1

teixeirak commented 4 years ago

@beckybanbury, the appropriate null to test whether the interaction is significant would be: mod.int: mean ~ mat + map + (1 | geographic.area/plot.name)

For the example above, I'd expect that it won't come out as significant, in which case we should present (graph) the additive model.

beckybanbury commented 4 years ago

Thanks - this will change our results a little, so I will update as soon as possible.