ValentineHerr
commented
4 years ago Trying to answer each bullet points bellow. Let me know if that helps or if you think I am looking at it in the wrong way. If that is the case we can talk on the phone and see where things can be made more clear in the manuscript so that there is no confusion about what you tried to do.
We obviously need to change the wording.
Yes, it might be what is throwing me off, but if you are looking at several variables in your "univariate" approach then I don't understand the difference with your "multivariate" approach.
- provides individual tests of the significance of variables (confirms their significance and direction in isolation, thereby ensuring that their presence in top models is justifies)
--> I disagree with this approach unless you don't have enough data to look at more than one variable at a time. The only thing you are testing by looking at one variable at a time is "is variable X the only variable driving Y?". Remember that the direction of a coefficient can flip when adding other important variables and this is very likely to happen if that other variable is correlated with the first one, which seems to be the case with CP and H. So I would never discuss the sign of a coefficient for a less than optimal model, and it is actually wrong to do so if two correlated variables are included in that model.
- Identifies variables worth including in subsequent models, while testing predictions on others that don't make the cut for the top models (e.g., WD, SLA)
--> to me this is an exploratory phase that you do on your own but it does not have its place in a manuscript. To me, unless you need to pick what variable among 2 correlated variables you want to move on with, you don't need to do this since the multivariate model selection approach would get rid of variables that are not important.
- Compares relative importance of different variables (e.g., height vs canopy position; canopy position with and without height in model)
--> I don't understand how table 4 would show that. if you want to look at relative importance of different variable you could try other options like:
- look at the size of the coefficients (only works when the variables have been standardized, it doesn't work with categorical variables)
- look at response curves (e.g. plot Y ~ X1 at different values of X2 in the same figure)
- sum the AIC weights of the models where the variable occurs (but you need a balanced set of candidates for that), I have never done it so you would need to read about it.. this could be a good start) Also , if H and CP are correlated they should never appear in the same model. It is tricky though because CP is categorical and H is not. In those cases, it usually is best to use the continuous variable because it takes only one parameters vs the number of levels in the categorical one.
@ValentineHerr, do you still think Table 4 is superfluous?
--> Probably yes. I think it is a useful table for you, in your exploratory phase, to get a feel for what is going on and maybe to select a sub-sample of variables to test in your multivariate analysis (if you had too many to look at and/for had collinearity among them). But I don't think it has it place in a manuscript. Table 4 looks quite confusing to me regardless. I think you forgot to define what dAICc is in the manuscript. Also, what is "Y" in the null variable column? drought year? if yes, maybe you should also call it "Y" in the column "variable". If you keep it, you need to help the reader more in the legend.
mcgregorian1
teixeirak
@mcgregorian1 and @ValentineHerr
@ValentineHerr commented:
"I think I don’t get why you are separating “univariate” and “multivariate” analysis. In other words, why would you look at the results of a univariate model if you know there are other variables that make that model better? You know that the sign of a coefficient can switch when you add another variable. It is not worth looking at that coefficient until you’ve found the best model. If you change that, you can get rid of Table4."
We obviously need to change the wording. "Univariate" refers to tests of one independent variable at a time, whereas "multivariate" referred to the models with all factors we believed would be important.
This is a potentially very important comment, and I'm thinking it through... My initial response is that Table 4 plays some important roles, but I'm open to being convinced otherwise. In my mind, Table 4 has the following roles:
@ValentineHerr, do you still think Table 4 is superfluous?