How can I validate my modeling for future periods?

[Oliveira-D-R]

I ask because the AUC values made available in the modeling process are only for the modeling of the present time.

[jamiemkass]

Validating future projections, or projections in the present for areas without occurrence data, is difficult because you lack the occurrence data you need to calculate performance metrics that require them, like AUC. Other kinds of validation might include inspecting the prediction for environmental values outside those used for model training with MESS, for example, to determine areas of uncertainty. I guess a model that results in a prediction with big areas that have very negative MESS values might be considered "less valid" than another model that does not, but we might be stretching the term "validation" here.

[hannahlowens]

+1 to @jamiemkass. MESS (or MOP in KUENM) will show you in which areas your model predictions are most likely to fail. If you have enough occurrence data to do so, you could also try hindcasting the model and seeing how well it predicts occurrences at a past time point. Then you at least have a bit of an idea of how robust your model is through time.

[Oliveira-D-R]

A lot of thanks. You can tell how can i do this on Wallace?

Em seg, 26 de abr de 2021 06:30, Hannah Owens @.***> escreveu:

+1 to @jamiemkass https://github.com/jamiemkass. MESS (or MOP in KUENM https://github.com/marlonecobos/kuenm/) will show you in which areas your model predictions are most likely to fail. If you have enough occurrence data to do so, you could also try hindcasting the model and seeing how well it predicts occurrences at a past time point. Then you at least have a bit of an idea of how robust your model is through time.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wallaceEcoMod/wallace/issues/348#issuecomment-826673693, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATY6KZU3X5YH2TOXW6VSIQ3TKUXELANCNFSM43KV3ISQ .

[gepinillab]

To run MESS, you first need to project your model (in area or time). Then, you just need to use the last module "Calculate Environmental Similarity" in the Project Component (or Tab).