Open pshipton opened 1 year ago
Not sure if this is Java 19 specific, or if it can be recreated, but adding to the Java 19 plan since it's discovered there.
@thallium please do a preliminary triage: try to reproduce the failure (locally or in a grinder), develop a theory and probable cause for it.
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/30426/ 100 iterations passed
@thallium Please try a 300x grinder with the below parameters to reproduce the issue. It will run the test 50 times on 6 different machines. Thus, increasing the chances to reproduce the failure.
TARGET=J9vmTest_0
PARALLEL=NodesByIterations
NUM_MACHINES=6
ITERATIONS=50
TKG_ITERATIONS=1
@babsingh https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/30458/ still unable to reproduce, not sure if I'm doing anything wrong.
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/30458/ still unable to reproduce, not sure if I'm doing anything wrong.
Both grinders are run correctly. The second grinder only runs the test with the following JVM options: -Xjit -Xgcpolicy:gencon -Xnocompressedrefs
. The failure has either been fixed or very difficult to reproduce. @pshipton Shall we close this issue until it is seen again?
Ok, closing until it's seen again.
+++ j9vm.test.thread.FindDeadlockTest: +++
command: /home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/bin/java -Xjit -Xgcpolicy:gencon -Xnocompressedrefs -Xdump -Xms64m -Xmx64m -Xbootclasspath:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/s390x/default/jclSC180/vm.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/se-service.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/rt.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/resources.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/jsse.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/charsets.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/jce.jar -classpath /home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/../../jvmtest/functional/VM_Test/VM_Test.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/../TKG/lib/asm-all.jar j9vm.test.thread.FindDeadlockTest
Locker got java.lang.Object@f80d5664
Blocker Thread[b2,5,main] locking java.lang.Object@f80d5664
Blocker Thread[b1,5,main] locking java.lang.Object@f80d5664
Blocker Thread[b3,5,main] locking java.lang.Object@f80d5664
Blocker Thread[c1,5,main] locking Thread[b1,5,main]
Blocker Thread[c2,5,main] locking Thread[c1,5,main]
Blocker Thread[c3,5,main] locking Thread[c2,5,main]
Testing no deadlocks...Passed.
Thread[d2,5,main] locked java.lang.Object@ff2849c8
Thread[d2,5,main] locking java.lang.Object@341d4c4b
Thread[d2,5,main] locked java.lang.Object@341d4c4b
Thread[d1,5,main] locked java.lang.Object@341d4c4b
Thread[d1,5,main] locking java.lang.Object@ff2849c8
Thread[d1,5,main] locked java.lang.Object@ff2849c8
Testing simple deadlock...
findDeadlockedThreads returned
findDeadlockedThreadsAndObjects returned
Failed. Wrong threads found.
Thread[d4,5,main] locked java.lang.Object@9b530918
Thread[d3,5,main] locked java.lang.Object@398d3c5b
Thread[d4,5,main] locking java.lang.Object@398d3c5b
Thread[d3,5,main] locking java.lang.Object@9b530918
Testing disjoint simple deadlocks...
findDeadlockedThreads returned
Thread[d3,5,main]
Thread[d4,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d3,5,main] java.lang.Object@9b530918
Thread[d4,5,main] java.lang.Object@398d3c5b
Failed. Wrong deadlocked threads.
Thread[d6,5,main] locked java.lang.Object@e937e071
Thread[d5,5,main] locked java.lang.Object@30343a1d
Thread[d6,5,main] locking java.lang.Object@341d4c4b
Thread[d6,5,main] locked java.lang.Object@341d4c4b
Thread[d5,5,main] locking java.lang.Object@341d4c4b
Thread[d5,5,main] locked java.lang.Object@341d4c4b
Testing multiple deadlocks...
findDeadlockedThreads returned
Thread[d3,5,main]
Thread[d4,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d3,5,main] java.lang.Object@9b530918
Thread[d4,5,main] java.lang.Object@398d3c5b
Failed. Wrong deadlocked threads.
Blocker Thread[c1,5,main] locking java.lang.Object@341d4c4b
Blocker Thread[c1,5,main] got java.lang.Object@341d4c4b
Blocker Thread[c2,5,main] locking Thread[c1,5,]
Blocker Thread[c2,5,main] got Thread[c1,5,]
Blocker Thread[c3,5,main] locking Thread[c2,5,]
Blocker Thread[c3,5,main] got Thread[c2,5,]
Blocker Thread[c4,5,main] locking Thread[c3,5,]
Blocker Thread[c4,5,main] got Thread[c3,5,]
Testing chained deadlocks...
findDeadlockedThreads returned
Thread[d3,5,main]
Thread[d4,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d3,5,main] java.lang.Object@9b530918
Thread[d4,5,main] java.lang.Object@398d3c5b
Failed. Wrong deadlocked threads.
Thread[j1,5,main] locked Thread[j1,5,main]
Thread[j4,5,main] locked Thread[j4,5,main]
Thread[j3,5,main] locked Thread[j3,5,main]
Thread[j2,5,main] locked Thread[j2,5,main]
Thread[j3,5,main] locking Thread[j4,5,main]
Thread[j1,5,main] locking Thread[j2,5,main]
Thread[j2,5,main] locking Thread[j3,5,main]
Thread[j4,5,main] locking Thread[j1,5,main]
Testing indirect deadlocks...
findDeadlockedThreads returned
Thread[d3,5,main]
Thread[d4,5,main]
Thread[j1,5,main]
Thread[j2,5,main]
Thread[j3,5,main]
Thread[j4,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d3,5,main] java.lang.Object@9b530918
Thread[d4,5,main] java.lang.Object@398d3c5b
Thread[j1,5,main] Thread[j2,5,main]
Thread[j2,5,main] Thread[j3,5,main]
Thread[j3,5,main] Thread[j4,5,main]
Thread[j4,5,main] Thread[j1,5,main]
Failed. Wrong deadlocked threads.
THERE WERE TEST FAILURES
no-zero exit value: 255
*** Test FAILED *** (j9vm.test.thread.FindDeadlockTest)
https://openj9-jenkins.osuosl.org/job/Test_openjdk11_j9_sanity.functional_s390x_linux_aot_Personal/206
J9vmTest_5
-Xshareclasses:name=test_aot -Xscmx400M -Xscmaxaot256m -Xcompressedrefs -Xjit -Xgcpolicy:gencon
+++ j9vm.test.thread.FindDeadlockTest: +++
command: /home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_s390x_linux_aot_Personal_testList_1/openjdkbinary/j2sdk-image/bin/java -Xshareclasses:name=test_aot -Xscmx400M -Xscmaxaot256m -Xcompressedrefs -Xjit -Xgcpolicy:gencon -Xdump -Xms64m -Xmx64m -classpath /home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_s390x_linux_aot_Personal_testList_1/aqa-tests/TKG/../../jvmtest/functional/VM_Test/VM_Test.jar:/home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_s390x_linux_aot_Personal_testList_1/aqa-tests/TKG/../TKG/lib/asm-all.jar j9vm.test.thread.FindDeadlockTest
Locker got java.lang.Object@7fe2bd8a
Blocker Thread[b3,5,main] locking java.lang.Object@7fe2bd8a
Blocker Thread[b2,5,main] locking java.lang.Object@7fe2bd8a
Blocker Thread[b1,5,main] locking java.lang.Object@7fe2bd8a
Blocker Thread[c1,5,main] locking Thread[b1,5,main]
Blocker Thread[c2,5,main] locking Thread[c1,5,main]
Blocker Thread[c3,5,main] locking Thread[c2,5,main]
Testing no deadlocks...Passed.
Thread[d1,5,main] locked java.lang.Object@52dd3366
Thread[d2,5,main] locked java.lang.Object@30c78cef
Thread[d2,5,main] locking java.lang.Object@52dd3366
Thread[d1,5,main] locking java.lang.Object@30c78cef
Testing simple deadlock...
Passed.
Thread[d3,5,main] locked java.lang.Object@1809baea
Thread[d3,5,main] locking java.lang.Object@7632fff4
Thread[d3,5,main] locked java.lang.Object@7632fff4
Thread[d4,5,main] locked java.lang.Object@7632fff4
Thread[d4,5,main] locking java.lang.Object@1809baea
Thread[d4,5,main] locked java.lang.Object@1809baea
Testing disjoint simple deadlocks...
findDeadlockedThreads returned
Thread[d1,5,main]
Thread[d2,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d1,5,main] java.lang.Object@30c78cef
Thread[d2,5,main] java.lang.Object@52dd3366
Failed. Wrong deadlocked threads.
Thread[d6,5,main] locked java.lang.Object@ad9d8f50
Thread[d5,5,main] locked java.lang.Object@4e8129f7
Thread[d5,5,main] locking java.lang.Object@52dd3366
Thread[d6,5,main] locking java.lang.Object@52dd3366
Testing multiple deadlocks...
findDeadlockedThreads returned
Thread[d1,5,main]
Thread[d2,5,main]
Thread[d5,5,main]
Thread[d6,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d1,5,main] java.lang.Object@30c78cef
Thread[d2,5,main] java.lang.Object@52dd3366
Thread[d5,5,main] java.lang.Object@52dd3366
Thread[d6,5,main] java.lang.Object@52dd3366
Failed. Wrong deadlocked threads.
Blocker Thread[c1,5,main] locking java.lang.Object@52dd3366
Blocker Thread[c2,5,main] locking Thread[c1,5,main]
Blocker Thread[c3,5,main] locking Thread[c2,5,main]
Blocker Thread[c4,5,main] locking Thread[c3,5,main]
Testing chained deadlocks...
findDeadlockedThreads returned
Thread[d1,5,main]
Thread[d2,5,main]
Thread[d5,5,main]
Thread[d6,5,main]
Thread[c1,5,main]
Thread[c2,5,main]
Thread[c3,5,main]
Thread[c4,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d1,5,main] java.lang.Object@30c78cef
Thread[d2,5,main] java.lang.Object@52dd3366
Thread[d5,5,main] java.lang.Object@52dd3366
Thread[d6,5,main] java.lang.Object@52dd3366
Thread[c1,5,main] java.lang.Object@52dd3366
Thread[c2,5,main] Thread[c1,5,main]
Thread[c3,5,main] Thread[c2,5,main]
Thread[c4,5,main] Thread[c3,5,main]
Failed. Wrong deadlocked threads.
Thread[j2,5,main] locked Thread[j2,5,main]
Thread[j3,5,main] locked Thread[j3,5,main]
Thread[j4,5,main] locked Thread[j4,5,main]
Thread[j1,5,main] locked Thread[j1,5,main]
Thread[j1,5,main] locking Thread[j2,5,main]
Thread[j4,5,main] locking Thread[j1,5,main]
Thread[j2,5,main] locking Thread[j3,5,main]
Thread[j3,5,main] locking Thread[j4,5,main]
Testing indirect deadlocks...
findDeadlockedThreads returned
Thread[d1,5,main]
Thread[d2,5,main]
Thread[d5,5,main]
Thread[d6,5,main]
Thread[c1,5,main]
Thread[c2,5,main]
Thread[c3,5,main]
Thread[c4,5,main]
Thread[j1,5,main]
Thread[j2,5,main]
Thread[j3,5,main]
Thread[j4,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d1,5,main] java.lang.Object@30c78cef
Thread[d2,5,main] java.lang.Object@52dd3366
Thread[d5,5,main] java.lang.Object@52dd3366
Thread[d6,5,main] java.lang.Object@52dd3366
Thread[c1,5,main] java.lang.Object@52dd3366
Thread[c2,5,main] Thread[c1,5,main]
Thread[c3,5,main] Thread[c2,5,main]
Thread[c4,5,main] Thread[c3,5,main]
Thread[j1,5,main] Thread[j2,5,main]
Thread[j2,5,main] Thread[j3,5,main]
Thread[j3,5,main] Thread[j4,5,main]
Thread[j4,5,main] Thread[j1,5,main]
Failed. Wrong deadlocked threads.
THERE WERE TEST FAILURES
no-zero exit value: 255
*** Test FAILED *** (j9vm.test.thread.FindDeadlockTest)
https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/656/
J9vmTest_3 -Xjit -Xgcpolicy:balanced -Xnocompressedrefs
+++ j9vm.test.thread.FindDeadlockTest: +++
command: /home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/jdkbinary/j2sdk-image/bin/java -Xjit -Xgcpolicy:balanced -Xnocompressedrefs -Xdump -Xms64m -Xmx64m -Xbootclasspath:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/jdkbinary/j2sdk-image/jre/lib/s390x/default/jclSC180/vm.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/jdkbinary/j2sdk-image/jre/lib/se-service.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/jdkbinary/j2sdk-image/jre/lib/rt.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/jdkbinary/j2sdk-image/jre/lib/resources.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/jdkbinary/j2sdk-image/jre/lib/jsse.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/jdkbinary/j2sdk-image/jre/lib/charsets.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/jdkbinary/j2sdk-image/jre/lib/jce.jar -classpath /home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/../../jvmtest/functional/VM_Test/VM_Test.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.functional_s390x_linux_Nightly_testList_1/aqa-tests/TKG/../TKG/lib/asm-all.jar j9vm.test.thread.FindDeadlockTest
Locker got java.lang.Object@ee0643d1
Blocker Thread[b2,5,main] locking java.lang.Object@ee0643d1
Blocker Thread[b1,5,main] locking java.lang.Object@ee0643d1
Blocker Thread[b3,5,main] locking java.lang.Object@ee0643d1
Blocker Thread[c1,5,main] locking Thread[b1,5,main]
Blocker Thread[c2,5,main] locking Thread[c1,5,main]
Blocker Thread[c3,5,main] locking Thread[c2,5,main]
Testing no deadlocks...Passed.
Thread[d1,5,main] locked java.lang.Object@efdb656c
Thread[d2,5,main] locked java.lang.Object@6249422f
Thread[d2,5,main] locking java.lang.Object@efdb656c
Thread[d1,5,main] locking java.lang.Object@6249422f
Testing simple deadlock...
Passed.
Thread[d3,5,main] locked java.lang.Object@6e62bd8c
Thread[d3,5,main] locking java.lang.Object@c0f1b4a6
Thread[d3,5,main] locked java.lang.Object@c0f1b4a6
Thread[d4,5,main] locked java.lang.Object@c0f1b4a6
Thread[d4,5,main] locking java.lang.Object@6e62bd8c
Thread[d4,5,main] locked java.lang.Object@6e62bd8c
Testing disjoint simple deadlocks...
findDeadlockedThreads returned
Thread[d1,5,main]
Thread[d2,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d1,5,main] java.lang.Object@6249422f
Thread[d2,5,main] java.lang.Object@efdb656c
Failed. Wrong deadlocked threads.
Thread[d6,5,main] locked java.lang.Object@d5e6ae05
Thread[d5,5,main] locked java.lang.Object@4d8039be
Thread[d6,5,main] locking java.lang.Object@efdb656c
Thread[d5,5,main] locking java.lang.Object@efdb656c
Testing multiple deadlocks...
findDeadlockedThreads returned
Thread[d1,5,main]
Thread[d2,5,main]
Thread[d5,5,main]
Thread[d6,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d1,5,main] java.lang.Object@6249422f
Thread[d2,5,main] java.lang.Object@efdb656c
Thread[d5,5,main] java.lang.Object@efdb656c
Thread[d6,5,main] java.lang.Object@efdb656c
Failed. Wrong deadlocked threads.
Blocker Thread[c1,5,main] locking java.lang.Object@efdb656c
Blocker Thread[c2,5,main] locking Thread[c1,5,main]
Blocker Thread[c3,5,main] locking Thread[c2,5,main]
Blocker Thread[c4,5,main] locking Thread[c3,5,main]
Testing chained deadlocks...
findDeadlockedThreads returned
Thread[d1,5,main]
Thread[d2,5,main]
Thread[d5,5,main]
Thread[d6,5,main]
Thread[c1,5,main]
Thread[c2,5,main]
Thread[c3,5,main]
Thread[c4,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d1,5,main] java.lang.Object@6249422f
Thread[d2,5,main] java.lang.Object@efdb656c
Thread[d5,5,main] java.lang.Object@efdb656c
Thread[d6,5,main] java.lang.Object@efdb656c
Thread[c1,5,main] java.lang.Object@efdb656c
Thread[c2,5,main] Thread[c1,5,main]
Thread[c3,5,main] Thread[c2,5,main]
Thread[c4,5,main] Thread[c3,5,main]
Failed. Wrong deadlocked threads.
Thread[j3,5,main] locked Thread[j3,5,main]
Thread[j4,5,main] locked Thread[j4,5,main]
Thread[j1,5,main] locked Thread[j1,5,main]
Thread[j2,5,main] locked Thread[j2,5,main]
Thread[j3,5,main] locking Thread[j4,5,main]
Thread[j4,5,main] locking Thread[j1,5,main]
Thread[j1,5,main] locking Thread[j2,5,main]
Thread[j2,5,main] locking Thread[j3,5,main]
Testing indirect deadlocks...
findDeadlockedThreads returned
Thread[d1,5,main]
Thread[d2,5,main]
Thread[d5,5,main]
Thread[d6,5,main]
Thread[c1,5,main]
Thread[c2,5,main]
Thread[c3,5,main]
Thread[c4,5,main]
Thread[j1,5,main]
Thread[j2,5,main]
Thread[j3,5,main]
Thread[j4,5,main]
findDeadlockedThreadsAndObjects returned
Thread[d1,5,main] java.lang.Object@6249422f
Thread[d2,5,main] java.lang.Object@efdb656c
Thread[d5,5,main] java.lang.Object@efdb656c
Thread[d6,5,main] java.lang.Object@efdb656c
Thread[c1,5,main] java.lang.Object@efdb656c
Thread[c2,5,main] Thread[c1,5,main]
Thread[c3,5,main] Thread[c2,5,main]
Thread[c4,5,main] Thread[c3,5,main]
Thread[j1,5,main] Thread[j2,5,main]
Thread[j2,5,main] Thread[j3,5,main]
Thread[j3,5,main] Thread[j4,5,main]
Thread[j4,5,main] Thread[j1,5,main]
Failed. Wrong deadlocked threads.
THERE WERE TEST FAILURES
no-zero exit value: 255
*** Test FAILED *** (j9vm.test.thread.FindDeadlockTest)
https://openj9-jenkins.osuosl.org/job/Test_openjdk19_j9_sanity.functional_s390x_linux_Nightly_testList_1/74 - ub20-390-5 J9vmTest_0
-Xjit -Xgcpolicy:gencon -Xnocompressedrefs