Provide a better error message when limit on number of classes is reached in Confusion Matrix

h2oai / h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

http://h2o.ai

Apache License 2.0

6.94k stars 2k forks source link

Provide a better error message when limit on number of classes is reached in Confusion Matrix #15452

Open exalate-issue-sync[bot] opened 1 year ago

exalate-issue-sync[bot] commented 1 year ago

For classification problems with > 1000 classes, the model building will fail in metric calculation phase because Confusion Matrix has a limit on a maximum number of classes. An error will be shown to the user.

We should: 1) Improve the error to inform the user the limit can be increased by a system property. 2) Consider not calculation the metric and continue training (if possible). 3) Show the error much earlier.

exalate-issue-sync[bot] commented 1 year ago

Erin LeDell commented: [~accountid:557058:04659f86-fbfe-4d01-90c9-146c34df6ee6] Should we just remove this limit and not calculate/print the confusion matrix for >1000 classes by default? That would mean there is no limit on the # of classes for any algo (though there might be a hardcoded class limit in tree algos, but I'd have to check on that).

exalate-issue-sync[bot] commented 1 year ago

hasithjp commented 1 year ago

JIRA Issue Migration Info

Jira Issue: PUBDEV-6118 Assignee: New H2O Bugs Reporter: Michal Kurka State: Open Fix Version: N/A Attachments: Available (Count: 1) Development PRs: N/A

Attachments From Jira

Attachment Name: Screen Shot 2018-12-10 at 11.22.08 AM.png Attached By: Michal Kurka File Link:https://h2o-3-jira-github-migration.s3.amazonaws.com/PUBDEV-6118/Screen Shot 2018-12-10 at 11.22.08 AM.png