search

kite-sdk / kite

Kite SDK
http://kitesdk.org/docs/current/
Apache License 2.0
394 stars 265 forks source link

Out of date POM? #468

Open carlosparadis opened 7 years ago

carlosparadis commented 7 years ago

I could not find a mailing list. Kindly redirect me to a discussion list if one exist and the issue tracker is not appropriate for user questions.

I am trying to install the Kite-SDK using mvn install -Dhadoop.profile=2 since I have Hadoop 2+ "installed" and maven on my local machine (the binaries were added to the env. path).

I am getting stuck on the installation step of kite:

[INFO] Reactor Summary: [INFO] [INFO] Kite Development Kit ............................... SUCCESS [ 0.804 s] [INFO] Kite Hadoop Dependencies Module .................... SUCCESS [ 0.767 s] [INFO] Kite Hadoop Default Dependencies Module ............ SUCCESS [ 0.244 s] [INFO] Kite Hadoop Default Test Dependencies Module ....... SUCCESS [ 0.118 s] [INFO] Kite Hadoop-1 Dependencies Module .................. SUCCESS [ 0.113 s] [INFO] Kite Hadoop-1 Test Dependencies Module ............. SUCCESS [ 0.040 s] [INFO] Kite Hadoop CDH4 Dependencies Module ............... SUCCESS [ 0.079 s] [INFO] Kite Hadoop CDH4 Test Dependencies Module .......... SUCCESS [ 0.060 s] [INFO] Kite Hadoop CDH5 Dependencies Module ............... SUCCESS [ 0.150 s] [INFO] Kite Hadoop CDH5 Test Dependencies Module .......... SUCCESS [ 0.088 s] [INFO] Kite Hadoop Compatibility Module ................... SUCCESS [ 7.327 s] [INFO] Kite HBase Dependencies Module ..................... SUCCESS [ 0.034 s] [INFO] Kite HBase Default Dependencies Module ............. SUCCESS [ 0.128 s] [INFO] Kite HBase Default Test Dependencies Module ........ SUCCESS [ 0.071 s] [INFO] Kite HBase Hadoop-1 Dependencies Module ............ SUCCESS [ 0.057 s] [INFO] Kite HBase Hadoop-1 Test Dependencies Module ....... SUCCESS [ 0.031 s] [INFO] Kite HBase CDH4 Dependencies Module ................ SUCCESS [ 0.046 s] [INFO] Kite HBase CDH4 Test Dependencies Module ........... SUCCESS [ 0.032 s] [INFO] Kite HBase CDH5 Dependencies Module ................ SUCCESS [ 0.076 s] [INFO] Kite HBase CDH5 Test Dependencies Module ........... SUCCESS [ 0.047 s] [INFO] Kite Data Module ................................... SUCCESS [ 0.384 s] [INFO] Kite Data Core Module .............................. SUCCESS [02:11 min] [INFO] Kite Data Oozie Module ............................. SUCCESS [ 11.892 s] [INFO] Kite Data Hive Module .............................. SUCCESS [01:35 min] [INFO] Kite Data S3 Module ................................ SUCCESS [ 5.735 s] [INFO] Kite Data HBase Module ............................. SUCCESS [02:17 min] [INFO] Kite Data MapReduce Module ......................... SUCCESS [ 43.277 s] [INFO] Kite Data Crunch Module ............................ SUCCESS [04:03 min] [INFO] Kite Data Flume Module ............................. SUCCESS [ 7.773 s] [INFO] Kite Data Spark Module ............................. FAILURE [ 17.148 s] [INFO] Kite Application POM for CDH4 ...................... SKIPPED [INFO] Kite Application POM for CDH5 ...................... SKIPPED [INFO] Kite Application POM Modules ....................... SKIPPED (etc, I am omiting the rest since it is all skipped.) This is the generated error (Running the specified command with -e and -X:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project kite-data-spark: There are test failures. [ERROR] [ERROR] Please refer to /Users/carlos/hadoop/kite/kite-data/kite-data-spark/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project kite-data-spark: There are test failures.

Please refer to /Users/carlos/hadoop/kite/kite-data/kite-data-spark/target/surefire-reports for the individual test results. at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures.

Please refer to /Users/carlos/hadoop/kite/kite-data/kite-data-spark/target/surefire-reports for the individual test results. at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution(SurefireHelper.java:91) at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary(SurefirePlugin.java:320) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:892) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:755) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) ... 20 more [ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :kite-data-spark

This post was the only solution I've found so far. I've attempted to execute mvn dependency::tree as suggested, which may have fixed the issue (as it is now stuck on the CDH5 step:

[INFO] Kite Data Crunch Module ............................ SUCCESS [ 0.116 s] [INFO] Kite Data Flume Module ............................. SUCCESS [ 0.044 s] [INFO] Kite Data Spark Module ............................. SUCCESS [ 0.173 s] [INFO] Kite Application POM for CDH4 ...................... SUCCESS [ 42.860 s] [INFO] Kite Application POM for CDH5 ...................... FAILURE [01:18 min] [INFO] Kite Application POM Modules ....................... SKIPPED [INFO] Kite Maven Plugin .................................. SKIPPED [INFO] Kite Tools Module .................................. SKIPPED [INFO] Kite Tools Runtime Module .......................... SKIPPED [INFO] Kite Minicluster ................................... SKIPPED

The error now is:

[ERROR] Failed to execute goal on project kite-app-parent-cdh5: Could not resolve dependencies for project org.kitesdk:kite-app-parent-cdh5:pom:1.1.1-SNAPSHOT: Could not find artifact org.apache.parquet:parquet-avro:jar:1.5.0-cdh5.4.2 in com.cloudera.releases (https://repository.cloudera.com/artifactory/cloudera-repos/) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :kite-app-parent-cdh5

Exploring the URL provided, however, it appears to be out of date. Could someone clarify what is the best way to fix this? Which version should I point instead? Where do I edit it so it runs after the appropriate url?

Thanks.