SCBI-ForestGEO / McGregor_climate-sensitivity-variation

repository for linking the climate sensitity of tree growth (derived from cores) to functional traits
0 stars 0 forks source link

Forward steps #46

Closed mcgregorian1 closed 5 years ago

mcgregorian1 commented 5 years ago

Hi @teixeirak

Regarding the model averaging, we also asked Valentine about this and she mentioned that she wasn’t sure about using it because she doesn’t have experience with it, but she knows of it as a tactic.

That being said, Valentine was stressing the importance of not disregarding models that are <2 dAIC from the top model. Currently what I’ve been doing (and Becky?) is that we run the covariates against each other, and determine the top model from that with a dAIC of 0.00. Valentine was mentioning that I should really also be looking at the AIC weight of the different models (which are tied to the dAIC), and noting that I shouldn't be ignoring any models that are within the range of 2 dAIC from the top model. She mentioned that it's not best to always be looking only at the model parsing through averages or otherwise, and instead that it would be good to be looking at the response curves for each covariate against the response variable (what Becky is doing?), plus potentially looking at the variance. This way, I should be focusing more on explaining and fully understanding what each model is saying for myself, instead of letting the numbers dictate my understanding.

She also mentioned that having my marginal and conditional r2 values be close together is encouraging (meaning that the covariates are indeed responsible for variation), even if the conditional (overall) values are low.

I recognize that what Valentine is saying is important, and I put it down to my lack of stats background that I didn't realize this would be more of an issue to get around from the start, in terms of making sure I fully understood the models and being able to describe the response curves. I'm fine attempting to do this next week seeing as it's something that needs to get done in order to throw out the models that are within 2 dAIC, but I also am cognizant of my reality. Before I leave I have 7 work days, 2 of which will be doing dendrobands. That leaves me 5 days for the following

After that, there are two scenarios:

I will do the best I can to get done what I can in my time left, but I wanted to give you a heads-up given that I don't expect to have much time once I officially start in Raleigh. Sorry I didn't realize this would be an issue earlier.

teixeirak commented 5 years ago

@mcgregorian1, I don't think we're so far off with the models. We haven't been graphing response curves, but we have been paying attention to the coefficients all along, and they've been consistent (except for the ones thrown out because we had too many many variables in the model).

I think its worth revisiting the model options within 2 dAIC of the ones you've identified as top to see whether there are other candidate models that should be considered, particularly in our testing of H3.2. Predictor variables varied across droughts. Make a table containing all models withing 2 dAIC of the top model, which can go into the SI. We can then reference this table in the results and discussion sections.

teixeirak commented 5 years ago

I just added a table summarizing hypotheses, predictions, and results, which will go at the end of the introduction where hypotheses are introduced. Please see the table heading (manuscript file) and table. I think this will help to give a clear picture of where we need to go. Specifically, you'll need to look at the best full models to determine whether predictions are consistently or inconsistently supported.

The table is in .xlsx, which breaks our fully automated analysis/manuscript integration, but I think this is the easiest way to handle this.

mcgregorian1 commented 5 years ago

Hi @teixeirak, ok, I can definitely make that table. Apologies if that whole comment sounded defeatist; when Valentine was explaining all of this it sounded in my mind like I was much more behind than I thought.

Also I didn't see your earlier comment about chatting at the picnic until just now, sorry about that.

I like that xlsx table, thank you for making that!

The plan this week is to have Alyssa shadow me for dendroband stuff today (shouldn't take all day), and then Thursday I'll be out doing the last intraannual survey. Otherwise I should be in the office.

mcgregorian1 commented 5 years ago

I've made the table and it's not as bad as I thought. All years, 1966, and 1977 only have 3 top models that are <2 dAIC. 1999 however has 9. I've uploaded the table as top_models_dAIC and I'll look at it more after I've done some dendroband stuff.

teixeirak commented 5 years ago

Looks good from a quick look. I'll look at it more carefully soon.

mcgregorian1 commented 5 years ago

@teixeirak I found some hiccups along the way but I've finished filling out the detailed version of Table 1.

  1. The interaction of TWI and height was including each other as a separate independent variable, skewing the results. I removed them, and this yielded the interaction to be significant enough to be a candidate (see tested_traits_all. I added it to the best_models list.
  2. Table 1 is updated on the notes side. I'm doing dendrobands tomorrow, but my goal is to now use this detailed version to fill in the presentation table on the left. Feel free to edit it - any cells that are bolded are notes for you / to understand my process better.
  3. I deleted a couple of the "tested_traits_all-[covariate]" files because I incorporated them into the tested_traits_all.csv by itself.
teixeirak commented 5 years ago
1. The interaction of TWI and height was including each other as a separate independent variable, skewing the results. I removed them, and this yielded the interaction to be significant enough to be a candidate (see [tested_traits_all](https://github.com/SCBI-ForestGEO/McGregor_climate-sensitivity-variation/blob/master/manuscript/tables_figures/tested_traits_all.csv). I added it to the best_models list.

Actually, we do want both height and TWI in the null model for this test. The test is whether there is a significant interaction in addition to the individual effects of each.

mcgregorian1 commented 5 years ago

Ah my bad. I've fixed it and updated the tables - TWI*height interaction is not significant anymore when comparing against a null model of just TWI and height independently

teixeirak commented 5 years ago

Okay, good. Our conclusions stay the same.