docularxu
commented
6 years ago I had the same issue. My environment is: Spark: 2.2.1, Scala: 2.11.8 HiBench build commands: mvn -Psparkbench -Dmodules -PstructuredStreaming -Dspark=2.2 -Dscala=2.11 clean package Then, I run these:
bin/workloads/streaming/identity/prepare/genSeedDataset.sh bin/workloads/streaming/identity/prepare/dataGen.sh
Run the streaming application: bin/workloads/streaming/identity/spark/run.sh
- Failure: java.lang.ClassNotFoundException: com.intel.hibench.sparkbench.streaming.RunBench
So I switched to structuredstreaming: bin/workloads/streaming/identity/spark/run_structuredstreaming.sh
- this can run successfully. I see logs on screen rolling rolling...
Then I tried to Generate the report: bin/workloads/streaming/identity/common/metrics_reader.sh Please input the topic: identiy
After that step, I see the same error log as above.
Anybody has any suggestions?
hadoop@Master:/usr/local/HiBench-master/bin/workloads/streaming/identity/common$ ./metrics_reader.sh patching args= Parsing conf: /usr/local/HiBench-master/conf/hadoop.conf Parsing conf: /usr/local/HiBench-master/conf/hibench.conf Parsing conf: /usr/local/HiBench-master/conf/spark.conf Parsing conf: /usr/local/HiBench-master/conf/storm.conf Parsing conf: /usr/local/HiBench-master/conf/workloads/streaming/identity.conf probe sleep jar: /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar start MetricsReader bench SPARK_identity_1_5_50_1525338393731 identity Please input the topic:identity log4j:WARN No appenders could be found for logger (org.I0Itec.zkclient.ZkConnection). log4j:WARN No appenders could be found for logger (org.I0Itec.zkclient.ZkEventThread). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Starting MetricsReader for kafka topic: identity Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.NumberFormatE... at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.intel.hibench.common.streaming.metrics.KafkaCollector$$anonfun$1.apply(Kafka... at com.intel.hibench.common.streaming.metrics.KafkaCollector$$anonfun$1.apply(Kafka... at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at scala.collection.TraversableLike$class.map(TraversableLike.scala:245) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at com.intel.hibench.common.streaming.metrics.KafkaCollector.start(KafkaCollector.s... at com.intel.hibench.common.streaming.metrics.MetricsReader$.delayedEndpoint$com$in... at com.intel.hibench.common.streaming.metrics.MetricsReader$delayedInit$body.apply(... at scala.Function0$class.apply$mcV$sp(Function0.scala:40) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) at scala.App$$anonfun$main$1.apply(App.scala:76) at scala.App$$anonfun$main$1.apply(App.scala:76) at scala.collection.immutable.List.foreach(List.scala:381) at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder... at scala.App$class.main(App.scala:76) at com.intel.hibench.common.streaming.metrics.MetricsReader$.main(MetricsReader.sca... at com.intel.hibench.common.streaming.metrics.MetricsReader.main(MetricsReader.scala) Caused by: java.lang.NumberFormatException: For input string: "0 182.163.112.4,nbizr... at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at scala.collection.immutable.StringLike$class.toLong(StringLike.scala:251) at scala.collection.immutable.StringOps.toLong(StringOps.scala:30) at com.intel.hibench.common.streaming.metrics.FetchJob.call(FetchJob.scala:32) at com.intel.hibench.common.streaming.metrics.FetchJob.call(FetchJob.scala:24) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
finish MetricsReader bench