Open ethanwhite opened 6 years ago
One possibility is to use something like a Mantel test to determine if richness is generally getting more different from the last year of data. In someway this is overkill for richness (we could just fit a regression) but it would make it directly applicable to the multi-variate environmental change measure in #254.
@skmorgane and I are thinking of using all of the data to quantify change (not just training or test). The overarching question is - do we do well in cases where richness is changing a lot. Measuring change on the training data is really more about the amount of information available for in the training set (an interesting question, but different from the overarching goal here). Measuring change on the test data focuses on "is there change during the forecasting window" which would probably be OK, but broadly we're interested in the overall dynamics of the system and so using all of the data seams initially like the most sensible way to go. The same reasoning applies to #254.
If richness isn't changing it's not surprising that baselines perform well, but what happens when richness is changing?