Outlier-resistant forecasting Performance Measures

[antoinecarme]

Add some outlier-resistant forecasting performance measures for robustness.

1. Some based on order statistics / ranks : Mann–Whitney U (equivalent to AUC), or Kendall rank correlation coefficient (Tau).

   https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test https://en.wikipedia.org/wiki/Kendall_rank_correlation_coefficient

2. Some based on signal distribution : Kolmogorov–Smirnov (KS).

   https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test#Kolmogorov%E2%80%93Smirnov_statistic

At least to have their value as a model debriefing indicator and to test their usage as a model selection criterion.

All these indicators have already an implementation in scipy. Easy.

[antoinecarme]

https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.mannwhitneyu.html

[antoinecarme]

https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.kendalltau.html

[antoinecarme]

https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.kstest.html

[antoinecarme]

Also add AUC

[antoinecarme]

Warning

PyAF models can be optimized/selected based on these measures ("KendallTau" , "KS", "MWU" and "AUC"), but this is very experimental and , of course, can generate strange models.

Higher values of these performance indicators are usually better (distribution values are compared).

Some tests :

tests/Kendall_KS_MWU_AUC_Perfs/test_ozone_Optimized_By_AUC.py
tests/Kendall_KS_MWU_AUC_Perfs/test_ozone_Optimized_By_Kendall_Tau.py
tests/Kendall_KS_MWU_AUC_Perfs/test_ozone_Optimized_By_KS.py
tests/Kendall_KS_MWU_AUC_Perfs/test_ozone_Optimized_By_Mann_Whittney_U.py

[antoinecarme]

FIXED