Cluster sizing guidance for binary/compressed files

h2oai / h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

http://h2o.ai

Apache License 2.0

6.9k stars 2k forks source link

Cluster sizing guidance for binary/compressed files #9184

Open exalate-issue-sync[bot] opened 1 year ago

exalate-issue-sync[bot] commented 1 year ago

Users find it difficult to estimate H2O cluster size for binary file formats like Parquet.

Our guidance is to allocate 4-5x memory of the "raw" size of the dataset. This is confusing to users of highly compressed files like Parquet.

h2o-ops commented 1 year ago

JIRA Issue Migration Info

Jira Issue: PUBDEV-6442 Assignee: New H2O Bugs Reporter: Michal Kurka State: Open Fix Version: N/A Attachments: N/A Development PRs: N/A