decide on performance assesment

[sdtaylor]

Many different ways to quantify the usefulness of an SDM.

• Rapacciuolo et al. 2012 compared compared only plots that changed between two time periods. He used the correct classification rate (same as precision).
• Rapocciulo et al 2014 conceived Temporal Validation Plots specifically for comparing SDM's across two time periods. They use all sample points available whether they changed or not between the two periods. They are heavily weighted toward points that do change though. They are a plot that is meant to be interpreted visually for individual spp, but can be summed up in custom functions for precision, correlation, and bias.
• Looking at any single metric (auc, precicion, ccr) over time is extremely noisy due to large environmental variation. Even at the larger temporal scales. This is potentially smoothed out by gradually offsetting the training years and averaging things out. see #4
• Looking at the observed changes of sites with high modeled probabilities of change between two time periods (extreme deltas) seems to gives a very intuitive feel for SDM performance over time.

[sdtaylor]

The fractions skill score used. Used in weather forecasting to find the best scale for probabilistic forecasts. Introduced in Scale-Selective Verification of Rainfall Accumulations from High-Resolution Forecasts of Convective Events

Also implemented in this branch https://github.com/sdtaylor/bbs-SDM/tree/probability-experiment

[sdtaylor]

FSS will be used.