Mllib 1.5 tests - Githubissues

databricks / spark-perf

Performance tests for Apache Spark

Apache License 2.0

380 stars 203 forks source link

Closed feynmanliang closed 9 years ago

feynmanliang commented 9 years ago

feynmanliang commented 9 years ago

@jkbradley Do you mind reviewing and merging this? It blocks #80

jkbradley commented 9 years ago

Did you compile this for 1.5 already? If that worked, then I'll merge this without careful checking, after fixing merge conflicts.

feynmanliang commented 9 years ago

Yep, this works but since 1.5 isn't an official release yet you will need to:

Set USE_CLUSTER_SPARK = True and MLLIB_SPARK_VERSION = 1.5 in config.py
Don't use PREP_MLLIB_TESTS = True; instead manually run cd mllib-tests; sbt/sbt -Dspark.version=1.5.0-SNAPSHOT clean assembly to build the mllib perf tests

These steps are necessary because we are building against snapshot. This should be fixed once we can build against an official 1.5 release.

jkbradley commented 9 years ago

I'll merge this, but in the next PR, can you please add those comments in doc somewhere? Thanks!