search

joons5834 / weather-forecast-accuracy

0 stars 0 forks source link

metrics for forecast performance(accuracy) #5

Open joons5834 opened 2 years ago

joons5834 commented 2 years ago

Goal: define the metrics for forecast performance for different data types

Different types of ground-truth data

joons5834 commented 2 years ago

https://www.metoffice.gov.uk/about-us/what/accuracy-and-trust/accuracy-performance https://openweathermap.org/accuracy-and-quality https://medium.com/human-in-a-machine-world/mae-and-rmse-which-metric-is-better-e60ac3bde13d https://towardsdatascience.com/comprehensive-guide-on-multiclass-classification-metrics-af94cfb83fbd

joons5834 commented 2 years ago

categorized values: Compare one classifier’s overall performance to another in a single metric — use Matthew’s correlation coefficient, Cohen’s kappa, and log loss. quantitative values: choose RMSE since it has the benefit of penalizing large errors compared to MAE. probability: F1 Score(Threshold value issue.) or AUC see https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc

joons5834 commented 2 years ago

Let users choose one vs. Provide only one metric?