search

linkedin / dr-elephant

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark
Apache License 2.0
1.35k stars 859 forks source link

Compilation Issue - Spark 2.3.0 and Hadoop - 2.7.3 #695

Closed userguy closed 4 years ago

userguy commented 4 years ago

[success] Total time: 0 s, completed Aug 10, 2020 2:16:54 PM [info] Updating {file:/home/drelephant/dr-elephant/}root... [info] Resolving org.apache.spark#spark-core_2.10;2.3.0 ...

error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2.10/2.3.0/spark-core_2.10-2.3.0.pom [warn] module not found: org.apache.spark#spark-core_2.10;2.3.0 [warn] ==== local: tried [warn] /home/hdfs/.ivy2/local/org.apache.spark/spark-core_2.10/2.3.0/ivys/ivy.xml [warn] ==== activator-launcher-local: tried [warn] /home/hdfs/drelephant/activator-dist-1.3.12/repository/org.apache.spark/spark-core_2.10/2.3.0/ivys/ivy.xml [warn] ==== public: tried [warn] https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.3.0/spark-core_2.10-2.3.0.pom [warn] ==== typesafe-releases: tried [warn] https://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2.10/2.3.0/spark-core_2.10-2.3.0.pom [warn] ==== typesafe-ivy-releasez: tried [warn] https://repo.typesafe.com/typesafe/ivy-releases/org.apache.spark/spark-core_2.10/2.3.0/ivys/ivy.xml [warn] ==== Typesafe Releases Repository: tried [warn] http://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2.10/2.3.0/spark-core_2.10-2.3.0.pom [info] Resolving org.apache#apache;7 ... [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/apache/apache/7/apache-7.jar [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/apache/commons/commons-parent/17/commons-parent-17.jar [info] Resolving org.eclipse.jetty#jetty-parent;14 ... [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/eclipse/jetty/jetty-parent/14/jetty-parent-14.jar [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/mortbay/jetty/jetty-parent/10/jetty-parent-10.jar [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/mortbay/jetty/project/6.1.26/project-6.1.26.jar [info] Resolving net.java#jvnet-parent;1 ... [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/sun/jersey/jersey-project/1.9/jersey-project-1.9.jar [info] Resolving org.slf4j#slf4j-parent;1.7.10 ... [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/slf4j/slf4j-parent/1.7.10/slf4j-parent-1.7.10.jar [info] Resolving com.google#google;5 ... [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/google/google/5/google-5.jar [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/google/inject/guice-parent/3.0/guice-parent-3.0.jar [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/google/inject/extensions/extensions-parent/3.0/extensions-parent-3.0.jar [info] Resolving org.sonatype.forge#forge-parent;6 ... [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/org/sonatype/forge/forge-parent/6/forge-parent-6.jar [info] Resolving asm#asm-parent;3.1 ... [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/asm/asm-parent/3.1/asm-parent-3.1.jar [info] Resolving com.sun.jersey#jersey-project;1.9 ... [error] SERVER ERROR: Bad Gateway url=http://repo.typesafe.com/typesafe/releases/com/sun/jersey/contribs/jersey-contribs/1.9/jersey-contribs-1.9.jar [info] Resolving org.apache.hadoop#hadoop-yarn;2.7.3 ... [error] SERVER ERROR: Bad Gateway

ShubhamGupta29 commented 4 years ago

Hi @userguy , the dependecy is wrong as Spark-Core_2.3 is not compatible with Scala 2.10. Kindly try this branch: https://github.com/ShubhamGupta29/dr-elephant/tree/feature_spark2.3, I will merge this branch soon into master. This would work for you, just don't change any version as the default version for this branch is 2.3.4. I am closing this ticket, feel free to re-open if there is some issue.