RedisLabs / spark-redis

A connector for Spark that allows reading and writing to/from Redis cluster
BSD 3-Clause "New" or "Revised" License
940 stars 372 forks source link

mvn clean package -DskipTests error #302

Open f771216203 opened 3 years ago

f771216203 commented 3 years ago

When I run "mvn clean package -DskipTests", I got an error. I tried openjdk-8 and openjdk-11, but it not work. How can I fix it?

WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/maven/lib/guice.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release [INFO] Scanning for projects... [INFO] Inspecting build with total of 1 modules... [INFO] Installing Nexus Staging features: [INFO] ... total of 1 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin [INFO] [INFO] -------------------< com.redislabs:spark-redis_2.11 >------------------- [INFO] Building Spark-Redis 2.6.0-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ spark-redis_2.11 --- [INFO] Deleting /home/yian/Pyspark/spark-redis/target [INFO] [INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (default) @ spark-redis_2.11 --- [INFO] argLine set to -javaagent:/home/yian/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar=destfile=/home/yian/Pyspark/spark-redis/target/jacoco.exec [INFO] [INFO] --- build-helper-maven-plugin:1.10:add-source (add-scala-sources) @ spark-redis_2.11 --- [INFO] Source directory: /home/yian/Pyspark/spark-redis/src/main/scala added. [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ spark-redis_2.11 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile) @ spark-redis_2.11 --- [WARNING] Expected all dependencies to require Scala version: 2.11.12 [WARNING] com.redislabs:spark-redis_2.11:2.6.0-SNAPSHOT requires scala version: 2.11.12 [WARNING] com.redislabs:spark-redis_2.11:2.6.0-SNAPSHOT requires scala version: 2.11.12 [WARNING] org.scala-lang:scalap:2.11.12 requires scala version: 2.11.12 [WARNING] com.redislabs:spark-redis_2.11:2.6.0-SNAPSHOT requires scala version: 2.11.12 [WARNING] org.scala-lang:scala-compiler:2.11.12 requires scala version: 2.11.12 [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12 [WARNING] org.scala-lang.modules:scala-xml_2.11:1.0.5 requires scala version: 2.11.7 [WARNING] Multiple versions of scala libraries detected! [INFO] /home/yian/Pyspark/spark-redis/src/main/scala:-1: info: compiling [INFO] Compiling 31 source files to /home/yian/Pyspark/spark-redis/target/classes at 1619607790653 [ERROR] error: java.lang.NoClassDefFoundError: javax/tools/ToolProvider [INFO] at scala.reflect.io.JavaToolsPlatformArchive.iterator(ZipArchive.scala:301) [INFO] at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) [INFO] at scala.reflect.io.AbstractFile.foreach(AbstractFile.scala:92) [INFO] at scala.tools.nsc.util.DirectoryClassPath.traverse(ClassPath.scala:277) [INFO] at scala.tools.nsc.util.DirectoryClassPath.x$15$lzycompute(ClassPath.scala:299) [INFO] at scala.tools.nsc.util.DirectoryClassPath.x$15(ClassPath.scala:299) [INFO] at scala.tools.nsc.util.DirectoryClassPath.packages$lzycompute(ClassPath.scala:299) [INFO] at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:299) [INFO] at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:264) [INFO] at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:358) [INFO] at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:358) [INFO] at scala.collection.Iterator$class.foreach(Iterator.scala:891) [INFO] at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) [INFO] at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) [INFO] at scala.collection.AbstractIterable.foreach(Iterable.scala:54) [INFO] at scala.tools.nsc.util.MergedClassPath.packages$lzycompute(ClassPath.scala:358) [INFO] at scala.tools.nsc.util.MergedClassPath.packages(ClassPath.scala:353) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply$mcV$sp(SymbolLoaders.scala:269) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply(SymbolLoaders.scala:260) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply(SymbolLoaders.scala:260) [INFO] at scala.reflect.internal.SymbolTable.enteringPhase(SymbolTable.scala:235) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader.doComplete(SymbolLoaders.scala:260) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:211) [INFO] at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1535) [INFO] at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:256) [INFO] at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) [INFO] at scala.tools.nsc.Global.rootMirror(Global.scala:71) [INFO] at scala.tools.nsc.Global.rootMirror(Global.scala:39) [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257) [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257) [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390) [INFO] at scala.tools.nsc.Global$Run.(Global.scala:1242) [INFO] at scala.tools.nsc.Driver.doCompile(Driver.scala:31) [INFO] at scala.tools.nsc.MainClass.doCompile(Main.scala:23) [INFO] at scala.tools.nsc.Driver.process(Driver.scala:51) [INFO] at scala.tools.nsc.Driver.main(Driver.scala:64) [INFO] at scala.tools.nsc.Main.main(Main.scala) [INFO] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [INFO] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [INFO] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [INFO] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [INFO] at scala_maven_executions.MainHelper.runMain(MainHelper.java:164) [INFO] at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26) [INFO] java.lang.reflect.InvocationTargetException [INFO] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [INFO] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [INFO] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [INFO] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [INFO] at scala_maven_executions.MainHelper.runMain(MainHelper.java:164) [INFO] at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26) [ERROR] Caused by: java.lang.NoClassDefFoundError: javax/tools/ToolProvider [INFO] at scala.reflect.io.JavaToolsPlatformArchive.iterator(ZipArchive.scala:301) [INFO] at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) [INFO] at scala.reflect.io.AbstractFile.foreach(AbstractFile.scala:92) [INFO] at scala.tools.nsc.util.DirectoryClassPath.traverse(ClassPath.scala:277) [INFO] at scala.tools.nsc.util.DirectoryClassPath.x$15$lzycompute(ClassPath.scala:299) [INFO] at scala.tools.nsc.util.DirectoryClassPath.x$15(ClassPath.scala:299) [INFO] at scala.tools.nsc.util.DirectoryClassPath.packages$lzycompute(ClassPath.scala:299) [INFO] at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:299) [INFO] at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:264) [INFO] at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:358) [INFO] at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:358) [INFO] at scala.collection.Iterator$class.foreach(Iterator.scala:891) [INFO] at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) [INFO] at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) [INFO] at scala.collection.AbstractIterable.foreach(Iterable.scala:54) [INFO] at scala.tools.nsc.util.MergedClassPath.packages$lzycompute(ClassPath.scala:358) [INFO] at scala.tools.nsc.util.MergedClassPath.packages(ClassPath.scala:353) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply$mcV$sp(SymbolLoaders.scala:269) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply(SymbolLoaders.scala:260) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply(SymbolLoaders.scala:260) [INFO] at scala.reflect.internal.SymbolTable.enteringPhase(SymbolTable.scala:235) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader.doComplete(SymbolLoaders.scala:260) [INFO] at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:211) [INFO] at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1535) [INFO] at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:256) [INFO] at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) [INFO] at scala.tools.nsc.Global.rootMirror(Global.scala:71) [INFO] at scala.tools.nsc.Global.rootMirror(Global.scala:39) [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257) [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257) [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390) [INFO] at scala.tools.nsc.Global$Run.(Global.scala:1242) [INFO] at scala.tools.nsc.Driver.doCompile(Driver.scala:31) [INFO] at scala.tools.nsc.MainClass.doCompile(Main.scala:23) [INFO] at scala.tools.nsc.Driver.process(Driver.scala:51) [INFO] at scala.tools.nsc.Driver.main(Driver.scala:64) [INFO] at scala.tools.nsc.Main.main(Main.scala) [INFO] ... 6 more [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.470 s [INFO] Finished at: 2021-04-28T19:03:10+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile) on project spark-redis_2.11: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 240 (Exit value: 240) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

fe2s commented 3 years ago

Hi @f771216203 , please make sure you use JDK 8. The build ran successfully on CircleCI with openJDK 8. Also works fine on my laptop with Oracle JDK 8. Please check the java version in the output of mvn -version

fe2s commented 3 years ago

You can also build it with docker

docker run -it --rm --name my-project -v "$(pwd)":/root -w /root adoptopenjdk/maven-openjdk8:latest mvn clean package -DskipTests
f771216203 commented 3 years ago

Thank you so much. I found that I should change JAVA_HOME path and now it success now. So redis-spark can't build on openjdk-11 only on openjdk8?