search

databricks / spark-sql-perf

Apache License 2.0
586 stars 406 forks source link

[ML-3824] Add benchmarks to mllib-large.yaml for FPGrowth #151

Closed lu-wang-dl closed 6 years ago

lu-wang-dl commented 6 years ago

Benchmark for FPGrowth is added to mllib-large.yaml.

jkbradley commented 6 years ago

@ludatabricks and @mengxr I see you removed logging of metrics for additional methods like AssociationRules in this PR: https://github.com/databricks/spark-sql-perf/pull/144/files

This logging was added in https://github.com/databricks/spark-sql-perf/pull/139/files so that we could time things other than fit() and transform().

Was this a mistake, or was there a problem with that logging?

lu-wang-dl commented 6 years ago

@jkbradley. That is a mistake. I should leave a comment there. In the new schema, we only want to save the results of the new methods instead of the function. I will fix it in a separate PR.

jkbradley commented 6 years ago

Sure, I'll say this LGTM, and we can adjust running times later as needed. Thanks!