tdebatty
commented
6 years ago Hello,
Which version of spark-knn-graphs are you using?
Le ven. 22 déc. 2017 à 07:46, Mohsin Rasool notifications@github.com a écrit :
Hello,
I am getting AbstractMethodError: while computing the KNN Graph. Can you locate what I am missing? The same code was running nicely few months ago, I guess, something has changed and cause this issue.
java.lang.AbstractMethodError: info.debatty.spark.knngraphs.builder.LSHSuperBit$1.call(Ljava/lang/Object;)Ljava/lang/Iterable; at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$3$1.apply(JavaRDDLike.scala:149) at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$3$1.apply(JavaRDDLike.scala:149) at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371) at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
The error is raised at bold line, I also tried graph.first()
` // Configure LSHSuperBit graph builder LSHSuperBitSparseIntegerVector gbuilder = new LSHSuperBitSparseIntegerVector(); gbuilder.setK(10); gbuilder.setStages(2); gbuilder.setBuckets(10); // LSH hashing requires the dimensionality gbuilder.setDim(dim);
// By default, LSHSuperBit graph builder uses cosine similarity // but another similarity measure can be defined if needed... // Build the graph... JavaPairRDD<Node<SparseIntegerVector>, NeighborList> graph = gbuilder.computeGraph(nodes); **java.util.List<Tuple2<Node<SparseIntegerVector>, NeighborList>> rddList = graph.collect();**`
Thanks in advance.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tdebatty/spark-knn-graphs/issues/11, or mute the thread https://github.com/notifications/unsubscribe-auth/AA1SDM5vU5qhsNnqQf40cZPVVia1OENkks5tC1BagaJpZM4RKs4j .
mohsinrasool
Hello,
I am getting AbstractMethodError: while computing the KNN Graph. Can you locate what I am missing? The same code was running nicely few months ago, I guess, something has changed and cause this issue.
java.lang.AbstractMethodError: info.debatty.spark.knngraphs.builder.LSHSuperBit$1.call(Ljava/lang/Object;)Ljava/lang/Iterable; at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$3$1.apply(JavaRDDLike.scala:149) at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$3$1.apply(JavaRDDLike.scala:149) at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371) at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)The error is raised at bold line, I also tried
graph.first()` // Configure LSHSuperBit graph builder LSHSuperBitSparseIntegerVector gbuilder = new LSHSuperBitSparseIntegerVector(); gbuilder.setK(10); gbuilder.setStages(2); gbuilder.setBuckets(10); // LSH hashing requires the dimensionality gbuilder.setDim(dim);
`
Thanks in advance.