sekruse / rheem-examples

Simple example apps for Rheem including Wordcount, IND detection, and PageRank
Apache License 2.0
3 stars 3 forks source link

Run pagerank encountered a problem #3

Open Foristkirito opened 7 years ago

Foristkirito commented 7 years ago

Please give some more details to run the examples. I just used mvn package and the: mvn -e -X package exec:java -Dexec.mainClass=com.github.sekruse.pagerank.PageRank -Dexec.args="spark pagerank.txt 10" to run the pagerank example. but i got the following errors:

[com.github.sekruse.pagerank.PageRank.main()] INFO org.qcri.rheem.core.api.Job - StopWatch results:
* Optimization                            - 0:00:00.299
  * Prepare                               - 0:00:00.087
    * Prune&Isolate                       - 0:00:00.017
    * Transformations                     - 0:00:00.070
    * Sanity                              - 0:00:00.000
  * Cardinality&Load Estimation           - 0:00:00.161
    * Create OptimizationContext          - 0:00:00.007
    * Create CardinalityEstimationManager - 0:00:00.001
    * Push Estimation                     - 0:00:00.153
  * Create Initial Execution Plan         - 0:00:00.051
    * Enumerate                           - 0:00:00.049
* Execution                               - 0:00:00.000
* Post-processing                         - 0:00:00.001
  * Release Resources                     - 0:00:00.001
[WARNING]
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.qcri.rheem.core.api.exception.RheemException: Could not find a single execution plan.
    at org.qcri.rheem.core.optimizer.enumeration.PlanEnumerator.enumerate(PlanEnumerator.java:201)
    at org.qcri.rheem.core.api.Job.createInitialExecutionPlan(Job.java:288)
    at org.qcri.rheem.core.api.Job.doExecute(Job.java:185)
    at org.qcri.rheem.core.util.OneTimeExecutable.tryExecute(OneTimeExecutable.java:23)
    at org.qcri.rheem.core.util.OneTimeExecutable.execute(OneTimeExecutable.java:36)
    at org.qcri.rheem.core.api.Job.execute(Job.java:160)
    at org.qcri.rheem.core.api.RheemContext.execute(RheemContext.java:92)
    at org.qcri.rheem.api.PlanBuilder.buildAndExecute(PlanBuilder.scala:84)
    at org.qcri.rheem.api.DataQuanta.collect(DataQuanta.scala:601)
    at com.github.sekruse.pagerank.PageRank.apply(PageRank.scala:63)
    at com.github.sekruse.pagerank.PageRank$.main(PageRank.scala:90)
    at com.github.sekruse.pagerank.PageRank.main(PageRank.scala)
    ... 6 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.500 s
[INFO] Finished at: 2017-02-17T13:11:55+00:00
[INFO] Final Memory: 35M/642M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java (default-cli) on project pagerank: An exception occured while executing the Java class. null: InvocationTargetException: Could not find a single execution plan. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java (default-cli) on project pagerank: An exception occured while executing the Java class. null
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. null
    at org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
    ... 20 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.qcri.rheem.core.api.exception.RheemException: Could not find a single execution plan.
    at org.qcri.rheem.core.optimizer.enumeration.PlanEnumerator.enumerate(PlanEnumerator.java:201)
    at org.qcri.rheem.core.api.Job.createInitialExecutionPlan(Job.java:288)
    at org.qcri.rheem.core.api.Job.doExecute(Job.java:185)
    at org.qcri.rheem.core.util.OneTimeExecutable.tryExecute(OneTimeExecutable.java:23)
    at org.qcri.rheem.core.util.OneTimeExecutable.execute(OneTimeExecutable.java:36)
    at org.qcri.rheem.core.api.Job.execute(Job.java:160)
    at org.qcri.rheem.core.api.RheemContext.execute(RheemContext.java:92)
    at org.qcri.rheem.api.PlanBuilder.buildAndExecute(PlanBuilder.scala:84)
    at org.qcri.rheem.api.DataQuanta.collect(DataQuanta.scala:601)
    at com.github.sekruse.pagerank.PageRank.apply(PageRank.scala:63)
    at com.github.sekruse.pagerank.PageRank$.main(PageRank.scala:90)
    at com.github.sekruse.pagerank.PageRank.main(PageRank.scala)
    ... 6 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
sekruse commented 7 years ago

The org.qcri.rheem.core.api.exception.RheemException: Could not find a single execution plan. indicates that you did not specify enough platforms to provide all necessary execution operators for the plan. In fact, for the PageRank example, you need to use at least one of java-graph, graphchi, spark-graph or basic-graph in addition to spark. For instance

mvn -e -X package exec:java -Dexec.mainClass=com.github.sekruse.pagerank.PageRank -Dexec.args="spark,spark-graph pagerank.txt 10"

will employ GraphX to implement the actual PageRankOperator.

Foristkirito commented 7 years ago

Great, it works! Thank you for your replying!

sekruse commented 7 years ago

I will leave this issue open to provide some actual examples in the readme. Thanks for reporting!