search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 721 forks source link

More "Blocking operation is not allowed in CRIU single thread mode" #18514

Closed pshipton closed 10 months ago

pshipton commented 11 months ago

Previous issue https://github.com/eclipse-openj9/openj9/issues/18399

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17/57 cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_3

Testing: Restore dump options test with no dump options specified before checkpoint and -Xdump:dynamic required
Test start time: 2023/11/22 11:21:47 Eastern Standard Time
Running command: bash /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu/criuScript.sh /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/jdkbinary/j2sdk-image/bin/java " -Xgcpolicy:optthruput  -Xtrace:maximal=mt,methods=org/eclipse/openj9/criu/CRIUSupport.registerRestoreOptionsFile,output=beforeCheckpoint.txt -Xdump:dynamic" org.openj9.criu.OptionsFileTest dumpOptionsTestRequireDynamic 1
Time spent starting: 3 milliseconds
Time spent executing: 3812 milliseconds
Test result: FAILED
Output from test:
 [OUT] start running script
 [OUT] export GLIBC_TUNABLES=glibc.cpu.hwcaps=-XSAVEC,-XSAVE,-AVX2,-ERMS,-AVX,-AVX_Fast_Unaligned_Load
 [OUT] export LD_BIND_NOT=on
 [OUT] /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/jdkbinary/j2sdk-image/bin/java -XX:+EnableCRIUSupport  -Xgcpolicy:optthruput  -Xtrace:maximal=mt,methods=org/eclipse/openj9/criu/CRIUSupport.registerRestoreOptionsFile,output=beforeCheckpoint.txt -Xdump:dynamic -cp /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu/criu.jar org.openj9.criu.OptionsFileTest dumpOptionsTestRequireDynamic 1
 [OUT] Pre-checkpoint
 [OUT] Performing CRIUSupport.checkpointJVM(), current thread name: main, Wed Nov 22 11:21:47 EST 2023, System.currentTimeMillis(): 1700670107781, System.nanoTime(): 93706827466861
 [OUT] JVMDUMP034I User requested Java dump using '/home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/output_17006680935281/cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_3/javacore.20231122.112147.806771.0001.txt' through CRIUSingleThreadModeJVMCRIUException
 [OUT] JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/output_17006680935281/cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_3/javacore.20231122.112147.806771.0001.txt
 [OUT] java.lang.reflect.InvocationTargetException
 [OUT] JVMJITM044W Some or all compiled code in the code cache invalidated post restore.
 [OUT] JVMDUMP034I User requested Java dump using '/home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/output_17006680935281/cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_3/javacore.20231122.112150.806771.0002.txt' through CRIUSingleThreadModeJVMCRIUException
 [OUT] JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/output_17006680935281/cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_3/javacore.20231122.112150.806771.0002.txt
 [OUT] Exception in thread "main" java.lang.BootstrapMethodError: bootstrap method initialization exception
 [OUT]  at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:188)
 [OUT]  at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:315)
 [OUT]  at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:281)
 [OUT]  at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:271)
 [OUT]  at java.base/java.lang.invoke.MethodHandleResolver.resolveInvokeDynamic(MethodHandleResolver.java:201)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.clearInetAddressCache(CRIUSupport.java:768)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.J9InternalCheckpointHookAPI$J9InternalCheckpointHook.runHook(J9InternalCheckpointHookAPI.java:143)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.J9InternalCheckpointHookAPI.runHooks(J9InternalCheckpointHookAPI.java:98)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.J9InternalCheckpointHookAPI.runPostRestoreHooksSingleThread(J9InternalCheckpointHookAPI.java:115)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVMImpl(Native Method)
 [OUT]  at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVM(CRIUSupport.java:823)
 [OUT]  at org.openj9.criu.CRIUTestUtils.checkPointJVM(CRIUTestUtils.java:77)
 [OUT]  at org.openj9.criu.OptionsFileTest.dumpOptionsTestRequireDynamic(OptionsFileTest.java:295)
 [OUT]  at org.openj9.criu.OptionsFileTest.main(OptionsFileTest.java:69)
 [OUT] Caused by: org.eclipse.openj9.criu.JVMRestoreException: Blocking operation is not allowed in CRIU single thread mode.
 [OUT]  at java.base/jdk.internal.ref.PhantomCleanable.insert(PhantomCleanable.java:87)
 [OUT]  at java.base/jdk.internal.ref.PhantomCleanable.<init>(PhantomCleanable.java:68)
 [OUT]  at java.base/jdk.internal.ref.CleanerImpl$PhantomCleanableRef.<init>(CleanerImpl.java:164)
 [OUT]  at java.base/java.lang.ref.Cleaner.register(Cleaner.java:220)
 [OUT]  at java.base/java.lang.invoke.MethodHandleNatives$CallSiteContext.make(MethodHandleNatives.java:90)
 [OUT]  at java.base/java.lang.invoke.CallSite.<init>(CallSite.java:144)
 [OUT]  at java.base/java.lang.invoke.ConstantCallSite.<init>(ConstantCallSite.java:50)
 [OUT]  at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:262)
 [OUT]  at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:341)
 [OUT]  at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:134)
 [OUT]  ... 13 more
 [OUT] JVMDUMP039I Processing dump event "vmstop", detail "#0000000000000000" at 2023/11/22 11:21:50 - please wait.
 [OUT] JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/output_17006680935281/cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_3/javacore.20231122.112150.806771.0003.txt' in response to an event
 [OUT] JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/output_17006680935281/cmdLineTester_criu_nonPortableRestore_Xtrace_outputFile_3/javacore.20231122.112150.806771.0003.txt
 [OUT] JVMDUMP013I Processed dump event "vmstop", detail "#0000000000000000".
 [OUT] Removed test output files
 [OUT] finished script
 [ERR] /home/jenkins/workspace/Test_openjdk17_j9_sanity.functional_aarch64_linux_OpenJDK17_testList_0/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu/criuScript.sh: line 41: 806771 Killed                  $2 -XX:+EnableCRIUSupport $3 -cp "$1/criu.jar" $4 $5 $6 > testOutput 2>&1
>> Success condition was found: [Output match: Killed]
>> Required condition was found: [Output match: Pre-checkpoint]
>> Success condition was found: [Output match: Processing dump event "vmstop"]
>> Success condition was not found: [Output match: Processing dump event "throw", detail "java/lang/OutOfMemoryError"]
>> Success condition was found: [Output match: JVM requested Java dump]
>> Success condition was not found: [Output match: Post-checkpoint]
>> Failure condition was found: [Output match: org.eclipse.openj9.criu.JVMRestoreException]
>> Failure condition was not found: [Output match: Dump option unrecognized: -Xdump:nofailover]
>> Failure condition was not found: [Output match: CRIU is not enabled]
>> Failure condition was not found: [Output match: Operation not permitted]
>> Success condition was not found: [Output match: Thread pid mismatch]
>> Success condition was not found: [Output match: do not match expected]
>> Success condition was not found: [Output match: Unable to create a thread:]
>> Failure condition was not found: [Output match: Could not dump the JVM processes, err=-70]
>> Failure condition was found: [Output match: User requested Java dump using]
pshipton commented 11 months ago

@TobiAjila

tajila commented 11 months ago

@singh264 Please take a look

pshipton commented 11 months ago

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_sanity.functional_s390x_linux_aot_Personal_testList_1/176 cmdLineTester_criu_nonPortableRestore_4 Testing: Envvar test1

singh264 commented 10 months ago

This issue can be closed and it was fixed by:

The changes are in v0.43.0-release as well.