search

USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
411 stars 143 forks source link

Maven packaging poblem #4

Closed karanjeets closed 8 years ago

karanjeets commented 8 years ago

~/git_workspace/sparkler/target$ java -classpath sparkler-0.1-SNAPSHOT-jar-with-dependencies.jar edu.usc.irds.sparkler.pipeline.Crawler -m "local" -j "sparkler-job-1465179374801" -i 1 2016-06-05 19:26:41 WARN NativeCodeLoader:62 [main] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2016-06-05 19:26:42 ERROR SparkContext:95 [main] - Error initializing SparkContext. com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka.version' at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:124) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:145) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:151) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:159) at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:164) at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:206) at akka.actor.ActorSystem$Settings.(ActorSystem.scala:169) at akka.actor.ActorSystemImpl.(ActorSystem.scala:505) at akka.actor.ActorSystem$.apply(ActorSystem.scala:142) at akka.actor.ActorSystem$.apply(ActorSystem.scala:119) at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121) at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53) at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52) at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(U

thammegowda commented 8 years ago

Fixed in 43dc88838bd729469091de297b70c5f83860fef4

karanjeets commented 8 years ago

Now build is failing. Please see below:

mvn clean compile package assembly:single
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /Users/karanjeetsingh/git_workspace/sparkler/target/sparkler-0.1-SNAPSHOT.jar with /Users/karanjeetsingh/git_workspace/sparkler/target/sparkler-0.1-SNAPSHOT-shaded.jar
[INFO] 
[INFO] --- maven-assembly-plugin:2.2-beta-5:single (default-cli) @ sparkler ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 42.397 s
[INFO] Finished at: 2016-06-05T23:12:40-07:00
[INFO] Final Memory: 64M/1862M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-5:single (default-cli) on project sparkler: Error reading assemblies: No assembly descriptors found. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
~/git_workspace/sparkler$
karanjeets commented 8 years ago

Packaging properly with below. The issue can be closed. @thammegowda - Do we need to update the Wiki Page?

mvn clean compile package
karanjeets commented 8 years ago

Wiki Updated. Section: Build and Deploy