search

irvingc / dbscan-on-spark

An implementation of DBSCAN runing on top of Apache Spark
Apache License 2.0
183 stars 58 forks source link

bumping to spark 1.6.1 #4

Closed eliasah closed 8 years ago

eliasah commented 8 years ago

I've just updated spark version to 1.6.1. Since there is no, guide lines for contributions. I've just tested with mvn test which ended with success

INFO] Scanning for projects... [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building Distributed DBSCAN on Apache Spark 0.2.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ dbscan-on-spark_2.10 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /media/eliasah/Transcend/github/eliasah/dbscan-on-spark/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ dbscan-on-spark_2.10 --- [INFO] No sources to compile [INFO] [INFO] --- scala-maven-plugin:3.2.0:compile (default) @ dbscan-on-spark_2.10 --- [WARNING] Expected all dependencies to require Scala version: 2.10.4 [WARNING] com.irvingc.spark:dbscan-on-spark_2.10:0.2.0-SNAPSHOT requires scala version: 2.10.4 [WARNING] com.twitter:chill_2.10:0.5.0 requires scala version: 2.10.4 [WARNING] com.typesafe.akka:akka-remote_2.10:2.3.11 requires scala version: 2.10.4 [WARNING] com.typesafe.akka:akka-actor_2.10:2.3.11 requires scala version: 2.10.4 [WARNING] com.typesafe.akka:akka-slf4j_2.10:2.3.11 requires scala version: 2.10.4 [WARNING] org.apache.spark:spark-core_2.10:1.6.1 requires scala version: 2.10.5 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ dbscan-on-spark_2.10 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 2 resources [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ dbscan-on-spark_2.10 --- [INFO] No sources to compile [INFO] [INFO] --- scala-maven-plugin:3.2.0:testCompile (default) @ dbscan-on-spark_2.10 --- [WARNING] Expected all dependencies to require Scala version: 2.10.4 [WARNING] com.irvingc.spark:dbscan-on-spark_2.10:0.2.0-SNAPSHOT requires scala version: 2.10.4 [WARNING] com.twitter:chill_2.10:0.5.0 requires scala version: 2.10.4 [WARNING] com.typesafe.akka:akka-remote_2.10:2.3.11 requires scala version: 2.10.4 [WARNING] com.typesafe.akka:akka-actor_2.10:2.3.11 requires scala version: 2.10.4 [WARNING] com.typesafe.akka:akka-slf4j_2.10:2.3.11 requires scala version: 2.10.4 [WARNING] org.apache.spark:spark-core_2.10:1.6.1 requires scala version: 2.10.5 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ dbscan-on-spark_2.10 --- [INFO] [INFO] --- scalatest-maven-plugin:1.0:test (test) @ dbscan-on-spark_2.10 --- Discovery starting. Discovery completed in 361 milliseconds. Run starting. Expected test count is: 9 LocalDBSCANArcherySuite: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 16/06/10 17:20:16 DEBUG LocalDBSCANArchery: total: 3

  • should cluster DBSCANSuite: 16/06/10 17:20:17 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 16/06/10 17:20:17 WARN Utils: Your hostname, AH-PC resolves to a loopback address: 127.0.1.1; using 192.168.69.68 instead (on interface eth1) 16/06/10 17:20:17 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address 16/06/10 17:20:24 DEBUG LocalDBSCANArchery: total: 3
  • should cluster 16/06/10 17:20:26 DEBUG DBSCAN: Found partitions: 16/06/10 17:20:26 DEBUG DBSCAN: (DBSCANRectangle(-2.4000000953674316,0.6000000238418579,2.4000000953674316,2.4000000953674316),243) 16/06/10 17:20:26 DEBUG DBSCAN: (DBSCANRectangle(-2.4000000953674316,-0.6000000238418579,2.4000000953674316,0.6000000238418579),225) 16/06/10 17:20:26 DEBUG DBSCAN: (DBSCANRectangle(-0.6000000238418579,-2.4000000953674316,2.4000000953674316,-0.6000000238418579),142) 16/06/10 17:20:26 DEBUG DBSCAN: (DBSCANRectangle(-2.4000000953674316,-2.4000000953674316,-0.6000000238418579,-0.6000000238418579),139) 16/06/10 17:20:26 WARN ClosureCleaner: Expected a closure; got org.apache.spark.mllib.clustering.dbscan.DBSCANPoint$ 16/06/10 17:20:26 DEBUG DBSCAN: About to find adjacencies 16/06/10 17:20:26 INFO LocalDBSCANNaive: About to start fitting 16/06/10 17:20:26 INFO LocalDBSCANNaive: About to start fitting 16/06/10 17:20:26 INFO LocalDBSCANNaive: found: 1 clusters 16/06/10 17:20:26 INFO LocalDBSCANNaive: found: 3 clusters 16/06/10 17:20:26 INFO LocalDBSCANNaive: About to start fitting 16/06/10 17:20:26 INFO LocalDBSCANNaive: found: 2 clusters 16/06/10 17:20:26 INFO LocalDBSCANNaive: About to start fitting 16/06/10 17:20:26 INFO LocalDBSCANNaive: found: 1 clusters 16/06/10 17:20:26 DEBUG DBSCAN: About to find all cluster ids 16/06/10 17:20:26 DEBUG DBSCAN: Connected clusters Set((1,1), (3,1), (2,1)) 16/06/10 17:20:26 DEBUG DBSCAN: Connected clusters Set((1,3), (0,1)) 16/06/10 17:20:26 DEBUG DBSCAN: Connected clusters Set((2,2), (1,2)) 16/06/10 17:20:26 DEBUG DBSCAN: Global Clusters 16/06/10 17:20:26 DEBUG DBSCAN: ((3,1),1) 16/06/10 17:20:26 DEBUG DBSCAN: ((1,1),1) 16/06/10 17:20:26 DEBUG DBSCAN: ((1,3),2) 16/06/10 17:20:26 DEBUG DBSCAN: ((2,2),3) 16/06/10 17:20:26 DEBUG DBSCAN: ((0,1),2) 16/06/10 17:20:26 DEBUG DBSCAN: ((1,2),3) 16/06/10 17:20:26 DEBUG DBSCAN: ((2,1),1) 16/06/10 17:20:26 INFO DBSCAN: Total Clusters: 7, Unique: 3 16/06/10 17:20:26 DEBUG DBSCAN: About to relabel inner points 16/06/10 17:20:26 DEBUG DBSCAN: About to relabel outer points 16/06/10 17:20:26 DEBUG DBSCAN: Done
  • dbscan EvenSplitPartitionerSuite:
  • should find partitions
  • should find two splits DBSCANGraphSuite:
  • should return connected
  • should return doubly connected
  • should return none for vertex
  • should return none for unknown Run completed in 11 seconds, 297 milliseconds. Total number of tests run: 9 Suites: completed 5, aborted 0 Tests: succeeded 9, failed 0, canceled 0, ignored 0, pending 0 All tests passed. [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 15.300 s [INFO] Finished at: 2016-06-10T17:20:27+02:00 [INFO] Final Memory: 26M/1216M [INFO] ------------------------------------------------------------------------

Is that ok with you ?

irvingc commented 8 years ago

Thanks for doing this, sorry for the delay in answering.

eliasah commented 8 years ago

It's ok don't mention it ! I was just a little worried that the project was inactive. I'm listing it here :)