search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 721 forks source link

cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile JVMRestoreException #17780

Closed pshipton closed 1 year ago

pshipton commented 1 year ago

https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/10 cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_0

Test start time: 2023/07/11 22:51:32 Coordinated Universal Time
Running command: bash /home/jenkins/workspace/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu/criuScript.sh /home/jenkins/workspace/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu /home/jenkins/workspace/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/bin/java " -Xjit -XX:+CRIURestoreNonPortableMode  -Xtrace:maximal=mt,methods=org/eclipse/openj9/criu/CRIUSupport.registerRestoreOptionsFile,output=beforeCheckpoint.txt -Xdump:dynamic" org.openj9.criu.OptionsFileTest dumpOptionsTestRequireDynamic 1
Time spent starting: 3 milliseconds
Time spent executing: 5489 milliseconds
Test result: FAILED
Output from test:
 [OUT] start running script
 [OUT] export GLIBC_TUNABLES=glibc.cpu.hwcaps=-XSAVEC,-XSAVE,-AVX2,-ERMS,-AVX,-AVX_Fast_Unaligned_Load
 [OUT] export LD_BIND_NOT=on
 [OUT] /home/jenkins/workspace/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/bin/java -XX:+EnableCRIUSupport  -Xjit -XX:+CRIURestoreNonPortableMode  -Xtrace:maximal=mt,methods=org/eclipse/openj9/criu/CRIUSupport.registerRestoreOptionsFile,output=beforeCheckpoint.txt -Xdump:dynamic -cp /home/jenkins/workspace/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu/criu.jar org.openj9.criu.OptionsFileTest dumpOptionsTestRequireDynamic 1
 [OUT] Pre-checkpoint
 [OUT] Performing CRIUSupport.checkpointJVM(), current thread name: main, Tue Jul 11 22:51:32 UTC 2023, System.currentTimeMillis(): 1689115892633, System.nanoTime(): 1689115892630180136
 [OUT] org.eclipse.openj9.criu.JVMCheckpointException: Blocking operation is not allowed in CRIU single thread mode.
 [OUT] JVMJITM044W Some or all compiled code in the code cache invalidated post restore.
 [OUT] Exception in thread "main" org.eclipse.openj9.criu.JVMRestoreException: Blocking operation is not allowed in CRIU single thread mode.
 [OUT]  at java.base/java.lang.ref.ReferenceQueue.poll(ReferenceQueue.java:78)
 [OUT]  at java.base/java.lang.invoke.MethodType$ConcurrentWeakInternSet.expungeStaleElements(MethodType.java:1460)
 [OUT]  at java.base/java.lang.invoke.MethodType$ConcurrentWeakInternSet.get(MethodType.java:1420)
 [OUT]  at java.base/java.lang.invoke.MethodType.makeImpl(MethodType.java:392)
 [OUT]  at java.base/java.lang.invoke.MethodType.methodType(MethodType.java:373)
 [OUT]  at java.base/java.lang.invoke.MethodType.fromDescriptor(MethodType.java:1221)
 [OUT]  at java.base/java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:1197)
 [OUT]  at java.base/java.lang.invoke.MethodTypeHelper.vmResolveFromMethodDescriptorString(MethodTypeHelper.java:259)
 [OUT]  at java.base/java.lang.invoke.MethodHandleResolver.resolveInvokeDynamic(MethodHandleResolver.java:170)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.clearInetAddressCache(CRIUSupport.java:624)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.J9InternalCheckpointHookAPI$J9InternalCheckpointHook.runHook(J9InternalCheckpointHookAPI.java:116)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.J9InternalCheckpointHookAPI.runHooks(J9InternalCheckpointHookAPI.java:80)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.J9InternalCheckpointHookAPI.runPostRestoreHooks(J9InternalCheckpointHookAPI.java:97)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVMImpl(Native Method)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVM(CRIUSupport.java:677)
 [OUT]  at org.openj9.criu.CRIUTestUtils.checkPointJVM(CRIUTestUtils.java:77)
 [OUT]  at org.openj9.criu.OptionsFileTest.dumpOptionsTestRequireDynamic(OptionsFileTest.java:295)
 [OUT]  at org.openj9.criu.OptionsFileTest.main(OptionsFileTest.java:69)
 [OUT] JVMDUMP039I Processing dump event "vmstop", detail "#0000000000000000" at 2023/07/11 22:51:36 - please wait.
 [OUT] JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/output_16891127574897/cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_0/javacore.20230711.225136.2146485.0001.txt' in response to an event
 [OUT] JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/output_16891127574897/cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_0/javacore.20230711.225136.2146485.0001.txt
 [OUT] JVMDUMP013I Processed dump event "vmstop", detail "#0000000000000000".
 [OUT] Removed test output files
 [OUT] finished script
 [ERR] /home/jenkins/workspace/Test_openjdk21_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu/criuScript.sh: line 41: 2146485 Killed                  $2 -XX:+EnableCRIUSupport $3 -cp "$1/criu.jar" $4 $5 $6 > testOutput 2>&1
>> Success condition was found: [Output match: Killed]
>> Required condition was found: [Output match: Pre-checkpoint]
>> Success condition was found: [Output match: Processing dump event "vmstop"]
>> Success condition was not found: [Output match: Processing dump event "throw", detail "java/lang/OutOfMemoryError"]
>> Success condition was found: [Output match: JVM requested Java dump]
>> Success condition was not found: [Output match: Post-checkpoint]
>> Failure condition was found: [Output match: org.eclipse.openj9.criu.JVMRestoreException]
>> Failure condition was not found: [Output match: Dump option unrecognized: -Xdump:nofailover]
>> Failure condition was not found: [Output match: CRIU is not enabled]
>> Failure condition was not found: [Output match: Operation not permitted]
>> Success condition was not found: [Output match: Thread pid mismatch]
>> Success condition was not found: [Output match: do not match expected]
>> Success condition was not found: [Output match: Unable to create a thread:]
>> Failure condition was not found: [Output match: Could not dump the JVM processes, err=-70]
pshipton commented 1 year ago

@tajila fyi

JasonFengJ9 commented 1 year ago

1/50 failed at https://openj9-jenkins.osuosl.org/job/Grinder/lastBuild/console

09:28:30  openjdk version "21-internal" 2023-09-19
09:28:30  OpenJDK Runtime Environment (build 21-internal-adhoc.****.BuildJDK21s390xlinuxNightly)
09:28:30  Eclipse OpenJ9 VM (build master-4c75373dfe0, JRE 21 Linux s390x-64-Bit Compressed References 20230711_16 (JIT enabled, AOT enabled)
09:28:30  OpenJ9   - 4c75373dfe0
09:28:30  OMR      - bcc94e42952
09:28:30  JCL      - 73fae096864 based on jdk-21+30)
JasonFengJ9 commented 1 year ago

This is expected to be resolved by https://github.com/eclipse-openj9/openj9/pull/17885