06_Spark1-homework的Spark Source Compile问题

[Chenxin123008]

执行下面命令时出现错误，有跟我遇到一样问题的同学吗？ docker build -t test/spark-source-maven -f Dockerfile.Ubuntu.14.04 . pox文件是自动下载的，没办法修改它的dependencies啊！

[INFO] Spark Project Parent POM ........................... SUCCESS [02:53 min] [INFO] Spark Project Test Tags ............................ SUCCESS [ 31.258 s] [INFO] Spark Project Launcher ............................. SUCCESS [ 50.111 s] [INFO] Spark Project Networking ........................... SUCCESS [ 11.425 s] [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 9.538 s] [INFO] Spark Project Unsafe ............................... SUCCESS [ 20.887 s] [INFO] Spark Project Core ................................. FAILURE [06:55 min] [INFO] Spark Project Bagel ................................ SKIPPED [INFO] Spark Project GraphX ............................... SKIPPED [INFO] Spark Project Streaming ............................ SKIPPED [INFO] Spark Project Catalyst ............................. SKIPPED [INFO] Spark Project SQL .................................. SKIPPED [INFO] Spark Project ML Library ........................... SKIPPED [INFO] Spark Project Tools ................................ SKIPPED [INFO] Spark Project Hive ................................. SKIPPED [INFO] Spark Project Docker Integration Tests ............. SKIPPED [INFO] Spark Project REPL ................................. SKIPPED [INFO] Spark Project YARN Shuffle Service ................. SKIPPED [INFO] Spark Project YARN ................................. SKIPPED [INFO] Spark Project Assembly ............................. SKIPPED [INFO] Spark Project External Twitter ..................... SKIPPED [INFO] Spark Project External Flume Sink .................. SKIPPED [INFO] Spark Project External Flume ....................... SKIPPED [INFO] Spark Project External Flume Assembly .............. SKIPPED [INFO] Spark Project External MQTT ........................ SKIPPED [INFO] Spark Project External MQTT Assembly ............... SKIPPED [INFO] Spark Project External ZeroMQ ...................... SKIPPED [INFO] Spark Project External Kafka ....................... SKIPPED [INFO] Spark Project Examples ............................. SKIPPED [INFO] Spark Project External Kafka Assembly .............. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 11:55 min [INFO] Finished at: 2017-04-12T14:28:53+00:00 [INFO] Final Memory: 61M/335M [INFO] ------------------------------------------------------------------------ [WARNING] The requested profile "hadoop-2.7" could not be activated because it does not exist. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:2.4.1:shade (default) on project spark-core_2.10: Error creating shaded jar: Could not resolve following dependencies: [org.eclipse.jetty:jetty-io:jar:8.1.14.v20131031 (provided)]: Could not resolve dependencies for project org.apache.spark:spark-core_2.10:jar:1.6.3: Could not transfer artifact org.eclipse.jetty:jetty-io:jar:8.1.14.v20131031 from/to central (https://repo1.maven.org/maven2): Connection reset -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :spark-core_2.10

[LiuMing5489]

嗯，一样的错误。 感觉除了学习目的除外， 还是在原来的hadoop版或者hbase版docker上部署预编译的spark比较简单。

[Chenxin123008]

@LiuMing5489 不过Docker还是挺好用，就是还没用明白。

[LiuMing5489]

嗯那，很好用，很轻量，就是网络问题比较郁闷。 有兴趣一起学习啊。😊

[xenron]

我这边又测试了一下，貌似可以啊。。。

Maven版本 报了个警告，确认了一下，应该是spark 1.6和hadoop 2.7不匹配 Ask on the mailing list, not JIRA. There is no 2.7 profile for 1.6.x. Use the 2.6 profile. 今天修改成2.6，再试试看

Sbt版本

[Chenxin123008]

@xenron 了解，我再用hadoop2.6试试