basler 2016 discussion

[sdtaylor]

potential new analysis writeup

We draw heavily from Basler 2016, who did a comprehensive model comparison using the budburst date for six tree species collected over 40 years from 139-283 sites across central Europe. The best performing models, with a RMSE of 4-6 days, were fit and validated with data from only a single site. Another suite of models fit by "pooling" all site data into a single model had, at best, RMSE error values of 7-9 days among the 6 species. A third suite performed the worst, with RMSE values of 9-10 days, where models were fit using information from a single site, and used to make predictions across many sites.

Here we outline 4 scenarios of how the LTS and NPN datasets can be evaluated. A) LTS derived models used to predict LTS observations, B) LTS derived models used to predict NPN observations, C) NPN derived models used to predict NPN observations, and D) NPN derived models used to predict LTS observations. Scenario A is analogous to the best performing models in Basler 2016, thus we expect this to perform the best (ie. have the lowest error). Scenario B is analogous to the worst performing models in Basler 2016 and we expect them to perform the worst. Scenario C is analogous to the "pooling" models which had performance midway between the other two in Basler 2016. There is no equivalent comparison that matches scenario D here, but we expect this situation to perform the better than scenario C since the LTS observations have less variance than NPN observations.

Expected ranking regardless of model or species:

1. Scenario A
2. Scenario D
3. Scenario C
4. scenario B

[sdtaylor]

from ethan: Get a distribution of difference between difference scenarios, and compare it the the best case scenario from basler. "the basler line"

[sdtaylor]

I'm potentially keeping the scenario descriptions here, but dropping all the basler comparison stuff