search

NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
https://nvidia.github.io/spark-rapids
Apache License 2.0
783 stars 228 forks source link

[BUG] Failed to build with mvnd. #11140

Open liurenjie1024 opened 2 months ago

liurenjie1024 commented 2 months ago

Describe the bug

mvnd is a project to help improve build performance of maven project. But it failed to build this project.

Steps/Code to reproduce bug

mvnd clean install -DskipTests -Dbuildver=342 -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dmaven.scaladoc.skip=true -Dmaven.scalastyle.skip=true -Ddist.jar.compress=false

Expected behavior

Succeeded as mvn.

Environment details (please complete the following information)

Ubuntu 22.04.

Additional context

mvnd --version
Apache Maven Daemon (mvnd) 1.0.0 linux-amd64 native client (0cb7cbcbed6fad010b14dcd05acdc0a59685d88d)
Terminal: org.jline.terminal.impl.PosixSysTerminal with pty org.jline.terminal.impl.jni.linux.LinuxNativePty
Apache Maven 3.9.8 (36645f6c9b5079805ea5009217e36f2cffd34256)
Maven home: /home/rayliu/.sdkman/candidates/mvnd/1.0.0/mvn
Java version: 1.8.0_402, vendor: Amazon.com Inc., runtime: /home/rayliu/.sdkman/candidates/java/8.0.402-amzn/jre
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "6.5.0-35-generic", arch: "amd64", family: "unix"
Purged 1 log files (log available in /home/rayliu/.m2/mvnd/registry/1.0.0/purge-2024-07-04.log)
mattahrens commented 1 month ago

What is the full error when the mvnd command fails?

liurenjie1024 commented 1 month ago

It failed in last step:


[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Skipping rapids-4-spark-integration-tests_2.12
[INFO] This project has been banned from the build due to previous failures.
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for RAPIDS Accelerator for Apache Spark Root Project 24.08.0-SNAPSHOT:
[INFO]
[INFO] RAPIDS Accelerator for Apache Spark Root Project ... SUCCESS [  3.029 s]
[INFO] rapids-4-spark-jdk-profiles_2.12 ................... SUCCESS [  0.648 s]
[INFO] rapids-4-spark-shim-deps-parent_2.12 ............... SUCCESS [  0.719 s]
[INFO] rapids-4-spark-sql-plugin-api_2.12 ................. SUCCESS [ 11.628 s]
[INFO] RAPIDS Accelerator for Apache Spark SQL Plugin ..... SUCCESS [ 41.063 s]
[INFO] RAPIDS Accelerator for Apache Spark Shuffle Plugin . SUCCESS [  8.437 s]
[INFO] RAPIDS Accelerator for Apache Spark Scala UDF Plugin SUCCESS [ 11.024 s]
[INFO] RAPIDS Accelerator for Apache Spark Delta Lake 2.4.x Support SUCCESS [ 16.624 s]
[INFO] RAPIDS Accelerator for Apache Spark Aggregator ..... SUCCESS [  2.803 s]
[INFO] Data Generator ..................................... SUCCESS [ 12.089 s]
[INFO] RAPIDS Accelerator for Apache Spark Distribution ... SUCCESS [ 20.161 s]
[INFO] rapids-4-spark-integration-tests_2.12 .............. SKIPPED
[INFO] RAPIDS Accelerator for Apache Spark Tests .......... SUCCESS [  0.812 s]
[INFO] RAPIDS Accelerator for Apache Spark Tools Support .. FAILURE [  4.758 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:36 min (Wall Clock)
[INFO] Finished at: 2024-07-10T09:57:48+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:3.1.0:run (shimplify-shim-sources) on project rapids-4-spark-tools-support: An Ant BuildException has occured: Traceback (most recent call last):
[ERROR]   File "<script>", line 194, in <module>
[ERROR]   File "/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py", line 825, in __init__
[ERROR]   File "/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py", line 667, in __init__
[ERROR]   File "/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py", line 646, in _addHandlerRef
[ERROR] java.lang.IllegalMonitorStateException: attempt to unlock read lock, not locked by current thread
[ERROR]     at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.unmatchedUnlockException(ReentrantReadWriteLock.java:444)
[ERROR]     at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:428)
[ERROR]     at java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1341)
[ERROR]     at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:881)
[ERROR]     at org.python.modules._weakref.GlobalRef.createReaperThreadIfAbsent(GlobalRef.java:343)
[ERROR]     at org.python.modules._weakref.GlobalRef.newInstance(GlobalRef.java:235)
[ERROR]     at org.python.modules._weakref.ReferenceType.weakref___new__(ReferenceType.java:36)
[ERROR]     at org.python.modules._weakref.ReferenceType$exposed___new__.new_impl(Unknown Source)
[ERROR]     at org.python.core.PyType.invokeNew(PyType.java:1119)
[ERROR]     at org.python.core.PyType.type___call__(PyType.java:2399)
[ERROR]     at org.python.core.PyType.__call__(PyType.java:2389)
[ERROR]     at org.python.core.PyObject.__call__(PyObject.java:477)
[ERROR]     at org.python.core.PyObject.__call__(PyObject.java:481)
[ERROR]     at logging$py._addHandlerRef$33(/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py:646)
[ERROR]     at logging$py.call_function(/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py)
[ERROR]     at org.python.core.PyTableCode.call(PyTableCode.java:173)
[ERROR]     at org.python.core.PyBaseCode.call(PyBaseCode.java:134)
[ERROR]     at org.python.core.PyFunction.__call__(PyFunction.java:416)
[ERROR]     at logging$py.__init__$35(/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py:668)
[ERROR]     at logging$py.call_function(/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py)
[ERROR]     at org.python.core.PyTableCode.call(PyTableCode.java:173)
[ERROR]     at org.python.core.PyBaseCode.call(PyBaseCode.java:306)
[ERROR]     at org.python.core.PyBaseCode.call(PyBaseCode.java:126)
[ERROR]     at org.python.core.PyFunction.__call__(PyFunction.java:416)
[ERROR]     at org.python.core.PyMethod.__call__(PyMethod.java:139)
[ERROR]     at logging$py.__init__$50(/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py:828)
[ERROR]     at logging$py.call_function(/home/rayliu/.m2/repository/org/python/jython-standalone/2.7.3/jython-standalone-2.7.3.jar/Lib/logging/__init__.py)
[ERROR]     at org.python.core.PyTableCode.call(PyTableCode.java:173)
[ERROR]     at org.python.core.PyBaseCode.call(PyBaseCode.java:306)
[ERROR]     at org.python.core.PyBaseCode.call(PyBaseCode.java:197)
[ERROR]     at org.python.core.PyFunction.__call__(PyFunction.java:485)
[ERROR]     at org.python.core.PyMethod.instancemethod___call__(PyMethod.java:237)
[ERROR]     at org.python.core.PyMethod.__call__(PyMethod.java:228)
[ERROR]     at org.python.core.PyMethod.__call__(PyMethod.java:223)
[ERROR]     at org.python.core.Deriveds.dispatch__init__(Deriveds.java:20)
[ERROR]     at org.python.core.PyObjectDerived.dispatch__init__(PyObjectDerived.java:1112)
[ERROR]     at org.python.core.PyType.type___call__(PyType.java:2408)
[ERROR]     at org.python.core.PyType.__call__(PyType.java:2389)
[ERROR]     at org.python.core.PyObject.__call__(PyObject.java:446)
[ERROR]     at org.python.core.PyObject.__call__(PyObject.java:450)
[ERROR]     at org.python.pycode._pyx0.f$0(<script>:579)
[ERROR]     at org.python.pycode._pyx0.call_function(<script>)
[ERROR]     at org.python.core.PyTableCode.call(PyTableCode.java:173)
[ERROR]     at org.python.core.PyCode.call(PyCode.java:18)
[ERROR]     at org.python.core.Py.runCode(Py.java:1703)
[ERROR]     at org.python.core.__builtin__.eval(__builtin__.java:497)
[ERROR]     at org.python.core.__builtin__.eval(__builtin__.java:501)
[ERROR]     at org.python.util.PythonInterpreter.eval(PythonInterpreter.java:255)
[ERROR]     at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:57)
[ERROR]     at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31)
[ERROR]     at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
[ERROR]     at org.apache.tools.ant.util.optional.JavaxScriptRunner.evaluateScript(JavaxScriptRunner.java:154)
[ERROR]     at org.apache.tools.ant.util.optional.JavaxScriptRunner.executeScript(JavaxScriptRunner.java:86)
[ERROR]     at org.apache.tools.ant.taskdefs.optional.script.ScriptDef.executeScript(ScriptDef.java:348)
[ERROR]     at org.apache.tools.ant.taskdefs.optional.script.ScriptDefBase.execute(ScriptDefBase.java:52)
[ERROR]     at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:299)
[ERROR]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]     at java.lang.reflect.Method.invoke(Method.java:498)
[ERROR]     at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
[ERROR]     at org.apache.tools.ant.Task.perform(Task.java:350)
[ERROR]     at org.apache.tools.ant.Target.execute(Target.java:449)
[ERROR]     at org.apache.tools.ant.Target.performTasks(Target.java:470)
[ERROR]     at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1401)
[ERROR]     at org.apache.tools.ant.Project.executeTarget(Project.java:1374)
[ERROR]     at org.apache.maven.plugins.antrun.AntRunMojo.execute(AntRunMojo.java:297)
[ERROR]     at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:126)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:328)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:316)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:174)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.access$000(MojoExecutor.java:75)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor$1.run(MojoExecutor.java:162)
[ERROR]     at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute(DefaultMojosExecutionStrategy.java:39)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:159)
[ERROR]     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:105)
[ERROR]     at io.takari.maven.builder.smart.SmartBuilderImpl.buildProject(SmartBuilderImpl.java:206)
[ERROR]     at io.takari.maven.builder.smart.SmartBuilderImpl$ProjectBuildTask.run(SmartBuilderImpl.java:71)
[ERROR]     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[ERROR]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[ERROR]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[ERROR]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[ERROR]     at java.lang.Thread.run(Thread.java:750)
[ERROR] java.lang.IllegalMonitorStateException: java.lang.IllegalMonitorStateException: attempt to unlock read lock, not locked by current thread
[ERROR]
[ERROR] around Ant part ...<shimplify if="shimplify" />... @ 7:33 in /home/rayliu/Workspace/spark-rapids/tools/target/spark342/antrun/build-main.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -r