search

databricks / spark-sql-perf

Apache License 2.0
586 stars 407 forks source link

[ML-3583] Add benchmarks to mllib-large.yaml for featurization #152

Closed lu-wang-dl closed 6 years ago

lu-wang-dl commented 6 years ago

Benchmark for featurization is added to mllib-large.yaml. Cannot run QuantileDiscretizer with spark 2.3. Leave this as future work: https://databricks.atlassian.net/browse/ML-3869

jkbradley commented 6 years ago

Thanks for the updates! Some of the running times are < 100ms. I'd recommend pushing everything up to at least 30 sec.

Also, this branch still needs to be rebased.

jkbradley commented 6 years ago

LGTM Thanks! Merging with master