What do we define as "best model"

[mcgregorian1]

Not to make another issue, but we should clarify this. The table below shows the best models for each drought scenario as per our discussion in #112 (removing crown position).

Currently the idea is to only show a figure / table with the results for all years combined as per #113, and within that, only show the best model.

• Normally, we'd say dAIC=0 is the best model and leave it, but I want to point out that I've seen many places how if you're within <1 dAIC (or indeed, <0.5 dAIC), then those models can essentially be considered to be equal.
  • I don't think this changes anything for Rt, where our dAIC=0 for "all" has all the variables already, however I do note that 1999 should be row 12, as it includes more variables and is equal to row 11. This is mainly for consideration for anything we may say in the text.

This point comes up for arimaratio, where if we go by this other rule, the model we should present as best is row 3, not row 2.

@teixeirak thoughts? We either

• continue with "best" model is only that which dAIC=0, or
• say we took models <0.5 dAIC to be considered equal, and therefore here is the top model.

[mcgregorian1]

If we decide to still include all the years for the visreg plot, then this would be the difference between having 1999 represented by more than 1 variable or not. The first plot shows keeping with dAIC=0, and the second is expanding to be <0.5 dAIC

[ValentineHerr]

• I've seen many places how if you're within <1 dAIC (or indeed, <0.5 dAIC), then those models can essentially be considered to be equal

There is a misconception about this. Yes, 2 models that are NOT nested and have a very similar AIC can be considered equal. But if you are looking at 2 nested models, (one of them is the same as the other except it has one more variable), then, if the bigger model doesn't have a smaller AIC it is not supported.

So in your case, adding the interaction term makes the AIC worse, and eventhough the dAIC is not that big, it still tells you there is no support for the interaction term and you should not consider it as a model in the "best set of models"

Comparing 1 and 2, it shows that TLP improved the model enough (lowered the AIC) to compensate the 2 AIC penalty points of adding a parameter to the model. It is not a strong improvement (dAIC is not big at all), but by AIC rules, TLP improves the model.

So, to me me, you have one best model.

(I you had the same model as your best one but with TLP replaced by some other variable, and giving a similar AIC, that is when you would say tha the 2 models are equivalent).

[mcgregorian1]

Ohh ok. That makes a lot more sense. Thank you!

[teixeirak]

So, @ValentineHerr , we should delete all of the models highlighted below?

[mcgregorian1]

If I understand it correctly, I think based on these top models we should drop all except the best ones given the nested elements.

[ValentineHerr]

I don’t think it hurts to show it. I usually like to see these kind of table. It helps understanding what made the best model what it is, what models we tried (or at least the top x-number of them).

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Ian McGregor notifications@github.com Sent: Thursday, July 16, 2020 8:26:00 PM To: SCBI-ForestGEO/McGregor_climate-sensitivity-variation McGregor_climate-sensitivity-variation@noreply.github.com Cc: Herrmann, Valentine HerrmannV@si.edu; Mention mention@noreply.github.com Subject: Re: [SCBI-ForestGEO/McGregor_climate-sensitivity-variation] What do we define as "best model" (#114)

External Email - Exercise Caution

If I understand it correctly, I think based on these top models we should drop all except the best ones given the nested elements.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSCBI-ForestGEO%2FMcGregor_climate-sensitivity-variation%2Fissues%2F114%23issuecomment-659752498&data=02%7C01%7Cherrmannv%40si.edu%7C0882632cf8204bfcad6208d829e7fbfe%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C637305423622474221&sdata=eOfWX6ceZKz9RwEfF2a4vyyY5HItRo%2BR9wZZfGa2EXQ%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEWDCIJA6ZMYRZWCOWYRNXTR36LBRANCNFSM4O42MMOQ&data=02%7C01%7Cherrmannv%40si.edu%7C0882632cf8204bfcad6208d829e7fbfe%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C637305423622484217&sdata=msHSt1dESaOjhBJd5ROEDoHeMzUpoGnQB7RQIpO%2B2RI%3D&reserved=0.

[teixeirak]

okay, thanks.