search

lucidworks / spark-solr

Tools for reading data from Solr as a Spark RDD and indexing objects from Spark into Solr using SolrJ.
Apache License 2.0
445 stars 250 forks source link

Error indexing spark stream into solr #120

Closed theyaa closed 7 years ago

theyaa commented 7 years ago

I appreciate your help in resolving this exception. I am doing a fairly simple spark stream indexing into solr and I can't resolve the exception below.

My Setup is: Spark 1.5.1 Solr 5.1.0 Spark-Solr 1.1.2

The code line causing exception is:

  SolrSupport.indexDStreamOfDocs(<kafka.zookeeper.quorum>, "stream_collection", batchSize, docs);

Below is the Stack Trace.

java.lang.VerifyError: Bad type on operand stack Exception Details: Location: org/apache/solr/common/util/JavaBinCodec.unmarshal(Ljava/io/InputStream;)Ljava/lang/Object; @71: invokevirtual Reason: Type 'org/apache/solr/common/util/FastInputStream' (current frame, stack[1]) is not assignable to 'org/apache/solr/common/util/DataInputInputStream' Current Frame: bci: @71 flags: { } locals: { 'org/apache/solr/common/util/JavaBinCodec', 'java/io/InputStream', 'org/apache/solr/common/util/FastInputStream' } stack: { 'org/apache/solr/common/util/JavaBinCodec', 'org/apache/solr/common/util/FastInputStream' } Bytecode: 0x0000000: 2bb8 0016 4d2a 2cb6 0017 b500 182a b400 0x0000010: 18b2 0012 9f00 31bb 0019 59bb 001a 59b7 0x0000020: 001b 121c b600 1db2 0012 b600 1e12 1fb6 0x0000030: 001d 2ab4 0018 b600 1e12 20b6 001d b600 0x0000040: 21b7 0022 bf2a 2cb6 0023 b0
Stackmap Table: append_frame(@69,Object[#357])

at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:82)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:341)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1098)
at org.apache.solr.colrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:869)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:805)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at com.lucidworks.spark.SolrSupport.sendBatchToSolr(SolrSupport.java:238)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:220)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:210)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:88)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

17/01/17 10:16:05 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[Executor task launch worker-1,5,main] java.lang.VerifyError: Bad type on operand stack Exception Details: Location: org/apache/solr/common/util/JavaBinCodec.unmarshal(Ljava/io/InputStream;)Ljava/lang/Object; @71: invokevirtual Reason: Type 'org/apache/solr/common/util/FastInputStream' (current frame, stack[1]) is not assignable to 'org/apache/solr/common/util/DataInputInputStream' Current Frame: bci: @71 flags: { } locals: { 'org/apache/solr/common/util/JavaBinCodec', 'java/io/InputStream', 'org/apache/solr/common/util/FastInputStream' } stack: { 'org/apache/solr/common/util/JavaBinCodec', 'org/apache/solr/common/util/FastInputStream' } Bytecode: 0x0000000: 2bb8 0016 4d2a 2cb6 0017 b500 182a b400 0x0000010: 18b2 0012 9f00 31bb 0019 59bb 001a 59b7 0x0000020: 001b 121c b600 1db2 0012 b600 1e12 1fb6 0x0000030: 001d 2ab4 0018 b600 1e12 20b6 001d b600 0x0000040: 21b7 0022 bf2a 2cb6 0023 b0
Stackmap Table: append_frame(@69,Object[#357])

at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:82)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:341)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequudSolrClient.java:1098)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:869)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:805)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at com.lucidworks.spark.SolrSupport.sendBatchToSolr(SolrSupport.java:238)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:220)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:210)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:88)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

17/01/17 10:16:05 ERROR TaskSetManager: Task 0 in stage 1.0 failed 1 times; aborting job 17/01/17 10:16:05 ERROR JobScheduler: Error running job streaming job 1484666165000 ms.0 org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1, localhost): java.lang.VerifyError: Bad type on operand stack Exception Details: Location: org/apache/solr/common/util/JavaBinCodec.unmarshal(Ljava/io/InputStream;)Ljava/lang/Object; @71: invokevirtual Reason: Type 'org/apache/solr/common/util/FastInputStream' (current frame, stack[1]) is not assignable to 'org/apache/solr/common/util/DataInputInputStream' Current Frame: bci: @71 flags: { } locals: { 'org/apache/solr/common/util/JavaBinCodec', 'java/io/InputStream', 'org/apache/solr/common/util/FastInputStream' } stack: { 'org/apache/solr/common/util/JavaBinCodec', 'org/apache/solr/common/util/FastInputStream' } Bytecode: 0x0000000: 2bb8 0016 4d2a 2cb6 0017 b500 182a b400 0x0000010: 18b2 0012 9f00 31bb 0019 59bb 001a 59b7 0x0000020: 001b 121c b600 1db2 0012 b600 1e12 1fb6 0x0000030: 001d 2ab4 0018 b600 1e12 20b6 001d b600 0x0000040: 21b7 0022 bf2a 2cb6 0023 b0
Stackmap Table: append_frame(@69,Object[#357])

at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:82)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:341)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.colrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1098)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:869)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:805)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at com.lucidworks.spark.SolrSupport.sendBatchToSolr(SolrSupport.java:238)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:220)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:210)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:88)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1283) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1271) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1270) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scaection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1270) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:697) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1496) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1458) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1447) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:567) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1824) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1837) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1850) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1921) at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:902) at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:900) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) at org.apache.spark.rdd.RDD.withScope(RDD.scala:310) at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:900) at org.apache.spark.api.java.JavaRDDLike$class.foreachPartition(JavaRDDLike.scala:222) at org.apache.spark.api.java.AbstractJavaRDDLike.foreachPartition(JavaRDDLike.scala:47) at com.lucidworks.spark.SolrSupport.indexDocs(SolrSupport.java:209) at com.lucidworks.spark.SolrSupport$3.call(SolrSupport.java:162) at com.lucidworks.spark.SolrSupport$3.call(SolrSupport.java:160) at org.apache.spark.streaming.api.java.JavaDStreamLikfun$foreachRDD$1.apply(JavaDStreamLike.scala:315) at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:315) at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:631) at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:631) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:42) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:40) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:40) at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:399) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:40) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:40) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:40) at scala.util.Try$.apply(Try.scala:161) at org.apache.spark.streaming.scheduler.Job.run(Job.scala:34) at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:218) at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:218) at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:218) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57) at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:217) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.VerifyError: Bad type on operand stack Exception Details: Location: org/apache/solr/common/util/JavaBinCodec.unmarshal(Ljava/io/InputStream;)Ljava/lang/Object; @71: invokevirtual Reason: Type 'org/apache/solr/common/util/FastInputStream' (current frame, stack[1]) is not assignable to 'org/apache/solr/common/util/DataInputInputStream' Current Frame: bci: @71 flags: { } locals: { 'org/apache/solr/common/util/JavaBinCodec', 'java/io/InputStream', 'org/apache/solr/common/util/FastInputStream' } stack: { 'org/apache/solr/common/util/JavaBinCodec', 'org/apache/solr/common/util/FastInputStream' } Bytecode: 0x0000000: 2bb8 0016 4d2a 2cb6 0017 b500 182a b400 0x0000010: 18b2 0012 9f00 31bb 0019 59bb 001a 59b7 0x0000020: 001b 121c b600 1db2 0012 b600 1e12 1fb6 0x0000030: 001d 2ab4 0018 b600 1e12 20b6 001d b600 0x0000040: 21b7 0022 bf2a 2cb6 0023 b0
Stackmap Table: append_frame(@69,Object[#357])

at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:82)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:341)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1098)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:869)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestolrClient.java:805)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at com.lucidworks.spark.SolrSupport.sendBatchToSolr(SolrSupport.java:238)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:220)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:210)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:88)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
... 3 more

Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1, localhost): java.lang.VerifyError: Bad type on operand stack Exception Details: Location: org/apache/solr/common/util/JavaBinCodec.unmarshal(Ljava/io/InputStream;)Ljava/lang/Object; @71: invokevirtual Reason: Type 'org/apache/solr/common/util/FastInputStream' (current frame, stack[1]) is not assignable to 'org/apache/solr/common/util/DataInputInputStream' Current Frame: bci: @71 flags: { } locals: { 'org/apache/solr/common/util/JavaBinCodec', 'java/io/InputStream', 'org/apache/solr/common/util/FastInputStream' } stack: { 'org/apache/solr/common/util/JavaBinCodec', 'org/apache/solr/common/util/FastInputStream' } Bytecode: 0x0000000: 2bb8 0016 4d2a 2cb6 0017 b500 182a b400 0x0000010: 18b2 0012 9f00 31bb 0019 59bb 001a 59b7 0x0000020: 001b 121c b600 1db2 0012 b600 1e12 1fb6 0x0000030: 001d 2ab4 0018 b600 1e12 20b6 001d b600 0x0000040: 21b7 0022 bf2a 2cb6 0023 b0
Stackmap Table: append_frame(@69,Object[#357])

at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:82)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:341)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1098)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:869)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:805)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at com.lucidworks.spark.SolrSupport.sendBatchToSolr(SolrSupport.java:238)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:220)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:210)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:88)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1283) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1271) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1270) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1270) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:697) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1496) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1458) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1447) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:567) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1824) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1837) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1850) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1921) at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:902) at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:900) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) at org.apache.spark.rdd.RDD.withScope(RDD.scala:310) at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:900) at org.apache.spark.api.java.JavaRDDLike$class.foreachPartition(JavaRDDLike.scala:222) at org.apache.spark.api.java.AbstractJavaRDDLike.foreachPartition(JavaRDDLike.scala:47) at com.lucidworks.spark.SolrSupport.indexDocs(SolrSupport.java:209) at com.lucidworks.spark.SolrSupport$3.call(SolrSupport.java:162) at com.lucidworks.spark.SolrSupport$3.call(SolrSupport.java:160) at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:315) at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:315) at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:631) at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:631) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:42) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:40) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:40) at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:399) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:40) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(FStream.scala:40) at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:40) at scala.util.Try$.apply(Try.scala:161) at org.apache.spark.streaming.scheduler.Job.run(Job.scala:34) at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:218) at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:218) at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:218) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57) at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:217) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.VerifyError: Bad type on operand stack Exception Details: Location: org/apache/solr/common/util/JavaBinCodec.unmarshal(Ljava/io/InputStream;)Ljava/lang/Object; @71: invokevirtual Reason: Type 'org/apache/solr/common/util/FastInputStream' (current frame, stack[1]) is not assignable to 'org/apache/solr/common/util/DataInputInputStream' Current Frame: bci: @71 flags: { } locals: { 'org/apache/solr/common/util/JavaBinCodec', 'java/io/InputStream', 'org/apache/solr/common/util/FastInputStream' } stack: { 'org/apache/solr/common/util/JavaBinCodec', 'org/apache/solr/common/util/FastInputStream' } Bytecode: 0x0000000: 2bb8 0016 4d2a 2cb6 0017 b500 182a b400 0x0000010: 18b2 0012 9f00 31bb 0019 59bb 001a 59b7 0x0000020: 001b 121c b600 1db2 0012 b600 1e12 1fb6 0x0000030: 001d 2ab4 0018 b600 1e12 20b6 001d b600 0x0000040: 21b7 0022 bf2a 2cb6 0023 b0
Stackmap Table: append_frame(@69,Object[#357])

at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:82)
at org.apache.solt.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:95)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:105)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:341)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1098)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:869)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:805)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at com.lucidworks.spark.SolrSupport.sendBatchToSolr(SolrSupport.java:238)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:220)
at com.lucidworks.spark.SolrSupport$5.call(SolrSupport.java:210)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:222)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:902)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1850)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:88)
at org.apache.spark.executor.Executor$TaskRunner.rutor.scala:214)
... 3 more

17/01/17 10:16:05 ERROR ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver

theyaa commented 7 years ago

I am using Spark 1.5.1, Solr 5.1.0, and spark-solr 1.1.2

Here is part of my gradle build:

def scalaVersion = "2.10"

def solrVersion = "5.1.0"

def sparkVersion = "1.5.1"

dependencies {

compile

"org.apache.spark:spark-streaming_${scalaVersion}:${sparkVersion}"

compile

"org.apache.spark:spark-streaming-kafka_${scalaVersion}:${sparkVersion}"

compile "org.apache.spark:spark-core_${scalaVersion}:{sparkVersion}"

compile "org.apache.kafka:kafka_${scalaVersion}:0.9.0.1"

compile "com.cloudera:spark-hbase:0.0.2-clabs"

compile 'com.typesafe:config:1.2.1'

compile 'org.slf4j:slf4j-log4j12:1.7.9'

compile 'org.apache.avro:avro:1.7.7'

compile "com.twitter:bijection-core_${scalaVersion}:0.7.2"

compile "com.twitter:bijection-avro_${scalaVersion}:0.7.2"

compile "org.json4s:json4s-core_${scalaVersion}:3.2.10"

compile "org.json4s:json4s-jackson_${scalaVersion}:3.2.10"

compile "org.json4s:json4s-ast_2.10:3.2.10"

compile "org.apache.solr:solr-solrj:${solrVersion}"

compile "org.apache.solr:solr-common:1.3.0"

compile "com.lucidworks.spark:spark-solr:1.1.2"

}

Thanks

On Tue, Jan 17, 2017 at 12:24 PM, Kiran Chitturi notifications@github.com wrote:

@theyaa https://github.com/theyaa What version of Spark, Solr and spark-solr library are you using ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lucidworks/spark-solr/issues/120#issuecomment-273236908, or mute the thread https://github.com/notifications/unsubscribe-auth/AGInk-e9N6g8X-Hb-lSjP0M2jewXrjLgks5rTPlJgaJpZM4LlyhH .

theyaa commented 7 years ago

I would appreciate your help with this matter. Does anyone has any idea how to fix this issue please?

thelabdude commented 7 years ago

version is your solr server? seems like a mismatch between solrj and solr server? try upgrading to later version of Solr

kiranchitturi commented 7 years ago

https://github.com/lucidworks/spark-solr#version-compatibility