Clean up model metrics in R and Python APIs

h2oai / h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Apache License 2.0

6.87k stars 2k forks source link

Some issues (these need to be turned into separate JIRAs):

The print-out between R and Python is not the same.
The metrics available between R and Python are not the same (e.g. Python multinomial is missing some metrics like accuracy).
The R (and maybe Python) API forces you to specify thresholds for many utility metrics functions, but thresholds should default to NULL and it should use the "optimal" threshold by default if no threshold is passed.
We might need to add missing metrics to the back-end.

Multinomial should contain (add) the following:

accuracy
error
mean_per_class_accuracy
mean_per_class_error

h2oai / h2o-3

Clean up model metrics in R and Python APIs #11721