search

karlhigley / spark-neighbors

Spark-based approximate nearest neighbor search using locality-sensitive hashing
MIT License
104 stars 23 forks source link

Spark 2.0/Scala 2.11 compatibility #55

Open laurendeason opened 7 years ago

laurendeason commented 7 years ago

We are upgrading to spark 2.0 and require scala 2.11 compatibility for our spark code; I have been using some of the spark-neighbors functionality which appears to only have scala 2.10 compatibility. Any chance someone might be working on upgrading this?

HermioneZhang commented 7 years ago

How do you write your build.sbt? I cannot find this project in my build.sbt

RoelantStegmann commented 6 years ago

Indeed running into problems after upgrading my spark :)

stderr: Warning: Master yarn-cluster is deprecated since 2.0. Please use master "yarn" with specified deploy mode instead. Ivy Default Cache set to: /home/livy/.ivy2/cache The jars for the packages stored in: /home/livy/.ivy2/jars :: loading settings :: url = jar:file:/usr/hdp/2.6.3.40-13/spark2/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml com.github.karlhigley#spark-neighbors_2.10 added as a dependency :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0 confs: [default] found com.github.karlhigley#spark-neighbors_2.10;0.2.2 in central found org.scalanlp#breeze_2.10;0.12 in central found org.scalanlp#breeze-macros_2.10;0.12 in central found org.scalamacros#quasiquotes_2.10;2.0.0-M8 in central found org.scala-lang#scala-reflect;2.10.4 in central found com.github.fommil.netlib#core;1.1.2 in central found net.sourceforge.f2j#arpack_combined_all;0.1 in central found net.sf.opencsv#opencsv;2.3 in central found com.github.rwl#jtransforms;2.4.0 in central found junit#junit;4.8.2 in central found org.apache.commons#commons-math3;3.2 in central found org.spire-math#spire_2.10;0.7.4 in central found org.spire-math#spire-macros_2.10;0.7.4 in central found org.scalamacros#quasiquotes_2.10;2.0.0 in central found org.slf4j#slf4j-api;1.7.5 in central found com.chuusai#shapeless_2.10.4;2.0.0 in central