search

ECMWFCode4Earth / ml_drought

Machine learning to better predict and understand drought. Moving github.com/ml-clim
https://ml-clim.github.io/drought-prediction/
90 stars 18 forks source link

Review - Validation #100

Closed cvitolo closed 5 years ago

cvitolo commented 5 years ago

What scores do you plan to use and why? Do you only want to use shap values? If yes, why?

gabrieltseng commented 5 years ago

We are currently using RMSE and R^2 to measure the distance between our predictions and the true VHI values. These are relatively standard metrics in the field of machine learning (and indeed climate science).

We use R^2 because this is the metric used by drought monitoring authorities in Kenya when validating the models they currently use in production.

Shap values are not necessarily used for quantitative validation, rather they are used to interpret feature importance.

cvitolo commented 5 years ago

That's very clear, many thanks! Closing this issue now.