search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 721 forks source link

DaaLoadTest ClassCastException #18195

Closed pshipton closed 10 months ago

pshipton commented 1 year ago

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_special.system_aarch64_linux_Personal_testList_4/76 DaaLoadTest_daa2_special_5m_22 -XX:+UseCompressedOops -Xjit -Xgcpolicy:gencon -Xaggressive

DLT class net.openj9.test.decimals.TestED2PD Total: 24 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestPD2ED Total: 47 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestPD2UD Total: 9 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestUD2PD Total: 13 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestBD2PD2BD Total: 12 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestDecimalData Total: 31 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestED2PD Total: 24 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestPD2ED Total: 47 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestPD2UD Total: 9 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestUD2PD Total: 13 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestBD2PD2BD Total: 12 Fail: 16 Ignore: 0
DLT     FAILURE: testBigInteger(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@996f1afc
DLT     FAILURE: testBigInteger(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@4dba2c86
DLT     FAILURE: testSimpleString(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@9fe2dd5f
DLT     FAILURE: testSimpleString(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@49bac929
DLT     FAILURE: testIncorrectPrecisionBD2PD(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@59d372f7
DLT     FAILURE: testIncorrectPrecisionBD2PD(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@11e74b04
DLT     FAILURE: testDifferentOffset(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@69e3412a
DLT     FAILURE: testDifferentOffset(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@361bf24a
DLT     FAILURE: testDFPPath(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@27a0ae25
DLT     FAILURE: testDFPPath(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@949b1176
DLT     FAILURE: testDecimalUsingPrimitivesPath(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@c8ab1d2d
DLT     FAILURE: testDecimalUsingPrimitivesPath(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@aaabc0e1
DLT     FAILURE: testLowerThanZero(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@5b6695dd
DLT     FAILURE: testLowerThanZero(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@5b5a1aac
DLT     FAILURE: testBadPDtoBD(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@2f2560d7
DLT     FAILURE: testBadPDtoBD(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@3b4ae97f
DLT 
DLT class net.openj9.test.decimals.TestDecimalData Total: 31 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestED2PD Total: 24 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestPD2ED Total: 47 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestPD2UD Total: 9 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestUD2PD Total: 13 Fail: 0 Ignore: 0
DLT 
DLT class net.openj9.test.decimals.TestBD2PD2BD Total: 12 Fail: 24 Ignore: 0
DLT     FAILURE: testSimpleLongs(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@6a829800
DLT     FAILURE: testSimpleLongs(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@cdb7a938
DLT     FAILURE: testSimpleInt(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@1357f2fd
DLT     FAILURE: testSimpleInt(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@ba83fad4
DLT     FAILURE: testPadding(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@153f99f4
DLT     FAILURE: testPadding(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@f8480f38
DLT     FAILURE: testFifteenPrecNoOverflow(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@f8d46266
DLT     FAILURE: testFifteenPrecNoOverflow(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@8ac6fda7
DLT     FAILURE: testBigInteger(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@b298ccbc
DLT     FAILURE: testBigInteger(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@aa5d45
DLT     FAILURE: testSimpleString(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@1a30ce0b
DLT     FAILURE: testSimpleString(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@9aec1eef
DLT     FAILURE: testIncorrectPrecisionBD2PD(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@3a4b41f1
DLT     FAILURE: testIncorrectPrecisionBD2PD(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@d4a465ed
DLT     FAILURE: testDifferentOffset(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@b2a36ed6
DLT     FAILURE: testDifferentOffset(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@14def06c
DLT     FAILURE: testDFPPath(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@d2561926
DLT     FAILURE: testDFPPath(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@c977829
DLT     FAILURE: testDecimalUsingPrimitivesPath(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@2e123ea3
DLT     FAILURE: testDecimalUsingPrimitivesPath(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@9c12cfee
DLT     FAILURE: testLowerThanZero(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@80b62a8
DLT     FAILURE: testLowerThanZero(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@b66226a1
DLT     FAILURE: testBadPDtoBD(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@3699dba8
DLT     FAILURE: testBadPDtoBD(net.openj9.test.decimals.TestBD2PD2BD) - java.lang.ClassCastException@f66c5790

etc.
pshipton commented 11 months ago

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_special.system_aarch64_linux_Personal_testList_0/85 DaaLoadTest_all_special_5m_22 -XX:+UseCompressedOops -Xjit -Xgcpolicy:gencon -Xaggressive

DLT class net.openj9.test.binaryData.LongIntegerComparison Total: 1 Fail: 1 Ignore: 0
DLT     FAILURE: DoComps(net.openj9.test.binaryData.LongIntegerComparison) - java.lang.ClassCastException@d156a6fe
...
DLT class net.openj9.test.binaryData.LongIntegerComparison Total: 1 Fail: 1 Ignore: 0
DLT     FAILURE: DoComps(net.openj9.test.binaryData.LongIntegerComparison) - java.lang.ClassCastException@f14a8691
...
DLT class net.openj9.test.binaryData.LongIntegerComparison Total: 1 Fail: 1 Ignore: 0
DLT     FAILURE: DoComps(net.openj9.test.binaryData.LongIntegerComparison) - java.lang.ClassCastException@637b5fa8
...
DLT class net.openj9.test.binaryData.LongIntegerComparison Total: 1 Fail: 1 Ignore: 0
DLT     FAILURE: DoComps(net.openj9.test.binaryData.LongIntegerComparison) - java.lang.ClassCastException@b589884
...
DLT class net.openj9.test.binaryData.LongIntegerComparison Total: 1 Fail: 1 Ignore: 0
DLT     FAILURE: DoComps(net.openj9.test.binaryData.LongIntegerComparison) - java.lang.ClassCastException@d5c42423
knn-k commented 11 months ago

1 failure in a 20x Grinder job for DaaLoadTest_all_special_5m_22: https://openj9-jenkins.osuosl.org/job/Grinder/3079/

DLT testStarted : testBoundaries(net.openj9.test.binaryData.TestDouble2ByteArray)
DLT testFailure: testBoundaries(net.openj9.test.binaryData.TestDouble2ByteArray): java.lang.ClassCastException@be153333
DLT java.lang.IllegalArgumentException: java.lang.ClassCastException@be153333
DLT     at sun.reflect.GeneratedMethodAccessor3751.invoke(Unknown Source)
DLT     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
DLT     at java.lang.reflect.Method.invoke(Method.java:503)
DLT     at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)

(Many other exceptions follow)
Akira1Saitoh commented 10 months ago

I got dump files for DaaLoadTest_all_special_5m_22. https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/36496/

JVMDUMP055I Processing dump event "systhrow", detail "java/lang/ClassCastException", exception "net.openj9.test.PD2Primitive.TestPD2I incompatible with net.openj9.test.PDMoveShifts.TestPDMove" at 2023/12/01 01:39:31 - please wait.
JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17014100474685/DaaLoadTest_all_special_5m_22_ITER_9/20231201-013453-DaaLoadTest/results/core.20231201.013931.550222.0001.dmp' in response to an event
JVMDUMP055I Processing dump event "systhrow", detail "java/lang/ClassCastException", exception "net.openj9.test.PD2Primitive.TestPD2I incompatible with net.openj9.test.PDMoveShifts.TestPDMove" at 2023/12/01 01:39:31 - please wait.
JVMDUMP010I System dump written to /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17014100474685/DaaLoadTest_all_special_5m_22_ITER_9/20231201-013453-DaaLoadTest/results/core.20231201.013931.550222.0001.dmp
core file generated - /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17014100474685/DaaLoadTest_all_special_5m_22_ITER_9/20231201-013453-DaaLoadTest/results/core.20231201.013931.550222.0001.dmp
JVMDUMP032I JVM requested JIT dump using '/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17014100474685/DaaLoadTest_all_special_5m_22_ITER_9/20231201-013453-DaaLoadTest/results/jitdump.20231201.013931.550222.0003.dmp' in response to an event
JVMDUMP051I JIT dump occurred in 'load-4' thread 0x0000000000464D00
JVMDUMP053I JIT dump is recompiling org/junit/internal/runners/statements/InvokeMethod.evaluate()V
JVMDUMP053I JIT dump is recompiling org/junit/runners/BlockJUnit4ClassRunner.runChild(Lorg/junit/runners/model/FrameworkMethod;Lorg/junit/runner/notification/RunNotifier;)V
JVMDUMP053I JIT dump is recompiling org/junit/runners/Suite.runChild(Ljava/lang/Object;Lorg/junit/runner/notification/RunNotifier;)V
JVMDUMP053I JIT dump is recompiling org/junit/runner/JUnitCore.run(Lorg/junit/runner/Runner;)Lorg/junit/runner/Result;
JVMDUMP053I JIT dump is recompiling net/openj9/test/PDMoveShifts/ShiftTestRunner.runTests(I)Z
JVMDUMP053I JIT dump is recompiling net/adoptopenjdk/loadTest/adaptors/ArbitraryJavaAdaptor.executeTest()Lnet/adoptopenjdk/loadTest/adaptors/AdaptorInterface$ResultStatus;
JVMDUMP010I JIT dump written to /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17014100474685/DaaLoadTest_all_special_5m_22_ITER_9/20231201-013453-DaaLoadTest/results/jitdump.20231201.013931.550222.0003.dmp
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/ClassCastException".
JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17014100474685/DaaLoadTest_all_special_5m_22_ITER_9/20231201-013453-DaaLoadTest/results/javacore.20231201.013931.550222.0002.txt' in response to an event
javacore file generated - /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17014100474685/DaaLoadTest_all_special_5m_22_ITER_9/20231201-013453-DaaLoadTest/results/javacore.20231201.013931.550222.0002.txt
JVMDUMP010I Java dump written to /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17014100474685/DaaLoadTest_all_special_5m_22_ITER_9/20231201-013453-DaaLoadTest/results/javacore.20231201.013931.550222.0002.txt
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/ClassCastException".

The stacktrace for the thread "load-4" is as follows:

<464d00> *** BEGIN STACK WALK, flags = 00400001 walkThread = 4,607,232 ***
<464d00>    ITERATE_O_SLOTS
<464d00>    RECORD_BYTECODE_PC_OFFSET
<464d00> Initial values: walkSP = 0x000000000047D6B8, PC = 0x0000000000000001, literals = 0x0000000000000000, A0 = 0x000000000047D6D0, j2iFrame = 0x000000000047D758, decomp = 0x0000000000000000
<464d00> Generic special frame: bp = 0x000000000047D6D0, sp = 0x000000000047D6B8, pc = 0x0000000000000001, cp = 0x0000000000000000, arg0EA = 0x000000000047D6D0, flags = 0x0000000000000000
<464d00> J2I frame: bp = 0x000000000047D758, sp = 0x000000000047D6D8, pc = 0x0000FFFEF4030B9D, cp = 0x000000000052A5C0, arg0EA = 0x000000000047D770, flags = 0x0000000010000000
<464d00>    Method: sun/reflect/GeneratedMethodAccessor4802.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x000000000052A6E8
<464d00>    Bytecode index = 13
<464d00>    Using local mapper
<464d00>    Locals starting at 0x000000000047D770 for 0x0000000000000003 slots
<464d00>        I-Slot: a0[0x000000000047D770] = 0x00000000FE628888
<464d00>        I-Slot: a1[0x000000000047D768] = 0x00000000F3A0F448
<464d00>        O-Slot: a2[0x000000000047D760] = 0x00000000F3A0F8D0
<464d00>    JIT-J2I-RegisterMap[0x000000000047D700] = UDATA(0x000000000052A400) (jit_r21)
<464d00>    JIT-J2I-RegisterMap[0x000000000047D708] = UDATA(0x00000000F3909CF0) (jit_r22)
<464d00>    JIT-J2I-RegisterMap[0x000000000047D710] = UDATA(0x00000000F3909CE0) (jit_r23)
<464d00>    JIT-J2I-RegisterMap[0x000000000047D718] = UDATA(0x00000000004AA100) (jit_r24)
<464d00>    JIT-J2I-RegisterMap[0x000000000047D720] = UDATA(0x00000000000E2A00) (jit_r25)
<464d00>    JIT-J2I-RegisterMap[0x000000000047D728] = UDATA(0x00000000F0397E58) (jit_r26)
<464d00>    JIT-J2I-RegisterMap[0x000000000047D730] = UDATA(0x00000000F3A0F8D0) (jit_r27)
<464d00>    JIT-J2I-RegisterMap[0x000000000047D738] = UDATA(0x00000000F3A0F448) (jit_r28)
<464d00> JIT inline frame: bp = 0x000000000047D858, pc = 0x0000FFFF6F2595A8, unwindSP = 0x000000000047D760, cp = 0x000000000010D4D0, arg0EA = 0x0000000000000000, jitInfo = 0x0000FFFF34254478
<464d00>    Method: sun/reflect/DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x000000000010D568
<464d00>    Bytecode index = 6, inlineDepth = 5, PC offset = 0x000000000041C798
<464d00> JIT inline frame: bp = 0x000000000047D858, pc = 0x0000FFFF6F2595A8, unwindSP = 0x000000000047D760, cp = 0x0000000000064A30, arg0EA = 0x0000000000000000, jitInfo = 0x0000FFFF34254478
<464d00>    Method: java/lang/reflect/Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x0000000000065B00
<464d00>    Bytecode index = 58, inlineDepth = 4, PC offset = 0x000000000041B298
<464d00> JIT inline frame: bp = 0x000000000047D858, pc = 0x0000FFFF6F2595A8, unwindSP = 0x000000000047D760, cp = 0x00000000004A9EC0, arg0EA = 0x0000000000000000, jitInfo = 0x0000FFFF34254478
<464d00>    Method: org/junit/runners/model/FrameworkMethod$1.runReflectiveCall()Ljava/lang/Object; !j9method 0x00000000004A9F98
<464d00>    Bytecode index = 15, inlineDepth = 3, PC offset = 0x0000000000A529D8
<464d00> JIT inline frame: bp = 0x000000000047D858, pc = 0x0000FFFF6F2595A8, unwindSP = 0x000000000047D760, cp = 0x00000000004A96C0, arg0EA = 0x0000000000000000, jitInfo = 0x0000FFFF34254478
<464d00>    Method: org/junit/internal/runners/model/ReflectiveCallable.run()Ljava/lang/Object; !j9method 0x00000000004A9758
<464d00>    Bytecode index = 1, inlineDepth = 2, PC offset = 0x00000000009F5688
<464d00> JIT inline frame: bp = 0x000000000047D858, pc = 0x0000FFFF6F2595A8, unwindSP = 0x000000000047D760, cp = 0x000000000049C210, arg0EA = 0x0000000000000000, jitInfo = 0x0000FFFF34254478
<464d00>    Method: org/junit/runners/model/FrameworkMethod.invokeExplosively(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; !j9method 0x000000000049C998
<464d00>    Bytecode index = 10, inlineDepth = 1, PC offset = 0x000000000083B7A8
<464d00> JIT frame: bp = 0x000000000047D858, pc = 0x0000FFFF6F2595A8, unwindSP = 0x000000000047D760, cp = 0x0000000000493EF0, arg0EA = 0x000000000047D860, jitInfo = 0x0000FFFF34254478
<464d00>    Method: org/junit/internal/runners/statements/InvokeMethod.evaluate()V !j9method 0x00000000004941E0
<464d00>    Bytecode index = 10, inlineDepth = 0, PC offset = 0x0000000000001458
<464d00>    stackMap=0x0000FFFF34255396, slots=I16(0x0001) parmBaseOffset=I16(0x0008), parmSlots=U16(0x0001), localBaseOffset=I16(0xFFB8)
<464d00>    Described JIT args starting at 0x000000000047D860 for U16(0x0001) slots
<464d00>        O-Slot: : a0[0x000000000047D860] = 0x00000000F3A0F680
<464d00>    Described JIT temps starting at 0x000000000047D810 for IDATA(0x0000000000000009) slots
<464d00>        I-Slot: : t8[0x000000000047D810] = 0x00000000F3A0F8C0
<464d00>        I-Slot: : t7[0x000000000047D818] = 0x00000000F3A0F898
<464d00>        SA-Obj[47d820]
<464d00>            F-Slot[0x000000000047D828] = 0x00000000F3A0F448
<464d00>            F-Slot[0x000000000047D82C] = 0x00000000F3A0F8D0
<464d00>            F-Slot[0x000000000047D830] = 0x00000000F390A5E0
<464d00>        I-Slot: : t5[0x000000000047D828] = 0xF3A0F8D0F3A0F448
<464d00>        I-Slot: : t4[0x000000000047D830] = 0x00000000F390A5E0
<464d00>        I-Slot: : t3[0x000000000047D838] = 0x00000000F3A0F780
<464d00>        I-Slot: : t2[0x000000000047D840] = 0x00000000F3A0F830
<464d00>        I-Slot: : t1[0x000000000047D848] = 0x00000000F3A0F680
<464d00>        I-Slot: : t0[0x000000000047D850] = 0x00000000F3A0F680
<464d00>    JIT-RegisterMap = UDATA(0x0000000000000000)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D7B0] = UDATA(0x0000000000494000) (jit_r21)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D7B8] = UDATA(0x00000000F3909CF0) (jit_r22)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D7C0] = UDATA(0x00000000F3909CE0) (jit_r23)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D7C8] = UDATA(0x00000000004AA100) (jit_r24)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D7D0] = UDATA(0x00000000000E2A00) (jit_r25)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D7D8] = UDATA(0x00000000F0397E58) (jit_r26)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D7E0] = UDATA(0x00000000F390E3F0) (jit_r27)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D7E8] = UDATA(0x00000000F390D8A8) (jit_r28)
<464d00> JIT inline frame: bp = 0x000000000047D938, pc = 0x0000FFFF6F239978, unwindSP = 0x000000000047D860, cp = 0x0000000000486D90, arg0EA = 0x0000000000000000, jitInfo = 0x0000FFFF34A829B8
<464d00>    Method: org/junit/runners/ParentRunner.runLeaf(Lorg/junit/runners/model/Statement;Lorg/junit/runner/Description;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x00000000004877A8
<464d00>    Bytecode index = 17, inlineDepth = 1, PC offset = 0x000000000022B1D4
<464d00> JIT frame: bp = 0x000000000047D938, pc = 0x0000FFFF6F239978, unwindSP = 0x000000000047D860, cp = 0x00000000004908E0, arg0EA = 0x000000000047D950, jitInfo = 0x0000FFFF34A829B8
<464d00>    Method: org/junit/runners/BlockJUnit4ClassRunner.runChild(Lorg/junit/runners/model/FrameworkMethod;Lorg/junit/runner/notification/RunNotifier;)V !j9method 0x00000000004910E8
<464d00>    Bytecode index = 30, inlineDepth = 0, PC offset = 0x0000000000000668
<464d00>    stackMap=0x0000FFFF34A82FF6, slots=I16(0x0003) parmBaseOffset=I16(0x0008), parmSlots=U16(0x0003), localBaseOffset=I16(0xFFA0)
<464d00>    Described JIT args starting at 0x000000000047D940 for U16(0x0003) slots
<464d00>        O-Slot: : a2[0x000000000047D940] = 0x00000000F3909CE0
<464d00>        O-Slot: : a1[0x000000000047D948] = 0x00000000F390A5E0
<464d00>        O-Slot: : a0[0x000000000047D950] = 0x00000000F390A230
<464d00>    Described JIT temps starting at 0x000000000047D8D8 for IDATA(0x000000000000000C) slots
<464d00>        O-Slot: : t11[0x000000000047D8D8] = 0x00000000F390DAE8
<464d00>        O-Slot: : t10[0x000000000047D8E0] = 0x00000000F390A230
<464d00>        O-Slot: : t9[0x000000000047D8E8] = 0x00000000F3A0F680
<464d00>        I-Slot: : t8[0x000000000047D8F0] = 0x00000000F3A0F2C0
<464d00>        I-Slot: : t7[0x000000000047D8F8] = 0x00000000002E1300
<464d00>        I-Slot: : t6[0x000000000047D900] = 0x00000000F390CB28
<464d00>        I-Slot: : t5[0x000000000047D908] = 0x0000FFFF6F0E1AB0
<464d00>        I-Slot: : t4[0x000000000047D910] = 0x00000000F3A0F390
<464d00>        O-Slot: : t3[0x000000000047D918] = 0x00000000F3A0F848
<464d00>        I-Slot: : t2[0x000000000047D920] = 0x00000000F390A260
<464d00>        I-Slot: : t1[0x000000000047D928] = 0x00000000F390A260
<464d00>        I-Slot: : t0[0x000000000047D930] = 0x00000000F3A0F258
<464d00>    JIT-RegisterMap = UDATA(0x0000000000000000)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D880] = UDATA(0x00000000F390A260) (jit_r21)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D888] = UDATA(0x00000000F390A230) (jit_r22)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D890] = UDATA(0x00000000F3909CE0) (jit_r23)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D898] = UDATA(0x00000000F3A0F370) (jit_r24)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D8A0] = UDATA(0x00000000002E1300) (jit_r25)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D8A8] = UDATA(0x00000000F0397E58) (jit_r26)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D8B0] = UDATA(0x00000000F390E3F0) (jit_r27)
<464d00>    JIT-Frame-RegisterMap[0x000000000047D8B8] = UDATA(0x00000000F390D8A8) (jit_r28)
...

The first argument of Method: sun/reflect/GeneratedMethodAccessor4802.invoke is an instanceof TestPD2I class as the ClassCastException suggests.

<464d00>        I-Slot: a1[0x000000000047D768] = 0x00000000F3A0F448

> !j9object 0x00000000F3A0F448
!J9Object 0x00000000F3A0F448 {
        struct J9Class* clazz = !j9class 0x4BC800 // net/openj9/test/PD2Primitive/TestPD2I

The receiver object of org/junit/internal/runners/statements/InvokeMethod.evaluate()V holds the instance of TestPD2I class at target field.

> !j9object 0x00000000F3A0F680
!J9Object 0x00000000F3A0F680 {
        struct J9Class* clazz = !j9class 0x494000 // org/junit/internal/runners/statements/InvokeMethod
        Object flags = 0x00000000;
        I lockword = 0x00000000 (offset = 0) (java/lang/Object) <hidden>
        Lorg/junit/runners/model/FrameworkMethod; testMethod = !fj9object 0xf390a5e0 (offset = 4) (org/junit/internal/runners/statements/InvokeMethod)
        Ljava/lang/Object; target = !fj9object 0xf3a0f448 (offset = 8) (org/junit/internal/runners/statements/InvokeMethod)
}
Akira1Saitoh commented 10 months ago

Instances of the test classes are created in org/junit/runners/BlockJUnit4ClassRunner$1.runReflectiveCall. https://github.com/junit-team/junit4/blob/16228f3ccea3c6f1170488e0e268f3601d130f75/src/main/java/org/junit/runners/BlockJUnit4ClassRunner.java#L309

Looking at the core file took in https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/36496/, I find the code to create an instance of TestPD2I class.

0xffff6f04c120 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +424                 ||   12185c00 and       w0, w0, #0xffffff00
0xffff6f04c124 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +428                 ||   f151641f cmp       x0, #0x459, lsl #12
0xffff6f04c128 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +432                 ||   54000ba1 b.ne      #0xffff6f04c29c C>> +804 
0xffff6f04c12c {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +436                 ||   d2990001 mov       x1, #0xc800 CONST 0x4bc800 J9Class - {net/openj9/test/PD2Primitive/TestPD2I}
0xffff6f04c130 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +440                 ||   f2a00961 movk      x1, #0x4b, lsl #16
0xffff6f04c134 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +444                 ||   f9403260 ldr       x0, [x19, #0x60]
0xffff6f04c138 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +448                 ||   f9403663 ldr       x3, [x19, #0x68]
0xffff6f04c13c {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +452                 ||   91010004 add       x4, x0, #0x40
0xffff6f04c140 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +456                 ||   eb03009f cmp       x4, x3
0xffff6f04c144 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +460                 ||   540032ec b.gt      #0xffff6f04c7a0 C>> +2088 
0xffff6f04c148 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +464                 ||   f9808095 prfm      pstl3strm, [x4, #0x100]
0xffff6f04c14c {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +468                 ||   f9003264 str       x4, [x19, #0x60]
0xffff6f04c150 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +472                 ||   b9000001 str       w1, [x0]
0xffff6f04c154 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +476                 ||   aa0003f5 mov       x21, x0
0xffff6f04c158 {org/.../BlockJUnit4ClassRunner$1.runReflectiveCall} +480                 ||   d5033abf dmb       ishst <<< ^+2100
Akira1Saitoh commented 10 months ago

I tried another run of Grinder with options to get trace logs of org/junit/runners/BlockJUnit4ClassRunner$1.runReflectiveCall. https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/36861/

This time the method belongs to TestByteArray2LongNumBytes and the passed receiver is TestByteArray2Integer.

JVMDUMP055I Processing dump event "systhrow", detail "java/lang/ClassCastException", exception "net.openj9.test.binaryData.TestByteArray2Integer incompatible with net.openj9.test.binaryData.TestByteArray2LongNumBytes" at 2023/12/12 09:35:59 - please wait.

In the trace log of runReflectiveCall, there is a new node (n2043n) that creates instance of TestByteArray2Integer. It is guarded by ProfiledGuard (n2297n), which tests if the class of DelegatingConstructorAccessorImpl.delegate with sun/reflect/GeneratedConstructorAccessor10.

n2335n    BBStart <block_79> (freq 9632) (extension of previous block)                        [    0xffff0823c990] bci=[17,0,45] rc=0 vc=3599 vn=- li=79 udi=- nc=0
n2310n    NULLCHK on n1996n [#32]                                                             [    0xffff0823c1c0] bci=[17,1,45] rc=0 vc=3599 vn=- li=79 udi=- nc=1
n2295n      aloadi  <vft-symbol>[#276  Shadow] [flags 0x18607 0x0 ] (X!=0 )                   [    0xffff0823bd10] bci=[17,1,45] rc=2 vc=3599 vn=- li=79 udi=- nc=1 flg=0x4
n1996n        l2a (X>=0 )                                                                     [    0xffff08235fa0] bci=[17,1,45] rc=3 vc=3599 vn=- li=79 udi=- nc=1 flg=0x100
n4026n          iu2l                                                                          [    0xffff0876da20] bci=[17,1,45] rc=1 vc=3599 vn=- li=79 udi=- nc=1
n4025n            iloadi  sun/reflect/DelegatingConstructorAccessorImpl.delegate Lsun/reflect/ConstructorAccessorImpl;[#636  Shadow +8] [flags 0x80607 0x0 ]  [    0xffff0876d9d0] bci=[17,1,45]
 rc=1 vc=3599 vn=- li=79 udi=- nc=1
n3073n              ==>aRegLoad
n2278n    aRegStore x14  (X!=0 privatizedInlinerArg )                                         [    0xffff0823b7c0] bci=[17,1,45] rc=0 vc=3599 vn=- li=79 udi=- nc=1 flg=0x2004
n1996n      ==>l2a
n2297n    ifacmpne --> block_46 BBStart at n2293n (ProfiledGuard/VftTest )                    [    0xffff0823bdb0] bci=[18,0,-] rc=0 vc=3599 vn=- li=79 udi=- nc=3 flg=0x1020
n2295n      ==>aloadi
n2296n      aconst 0x459000 (sun/reflect/GeneratedConstructorAccessor10.class) (classPointerConstant X!=0 X>=0 X<=0 )  [    0xffff0823bd60] bci=[18,0,-] rc=1 vc=3599 vn=- li=79 udi=- nc=0 flg=0x10304
n3079n      GlRegDeps                                                                         [    0xffff0831b220] bci=[18,0,-] rc=1 vc=3599 vn=- li=79 udi=- nc=2
n3077n        PassThrough x14                                                                 [    0xffff0831b180] bci=[17,1,45] rc=1 vc=3599 vn=- li=79 udi=- nc=1
n1996n          ==>l2a
n3074n        ==>aRegLoad
n2290n    BBEnd </block_79>                                                                   [    0xffff0823bb80] bci=[18,0,-] rc=0 vc=3599 vn=- li=79 udi=- nc=0
n2298n    BBStart <block_45> (freq 9632) (extension of previous block)                        [    0xffff0823be00] bci=[18,8,-] rc=0 vc=3599 vn=- li=45 udi=- nc=0
n2044n    treetop                                                                             [    0xffff08236ea0] bci=[18,0,-] rc=0 vc=3599 vn=- li=45 udi=- nc=1
n2043n      new  jitNewObject[#91  helper Method] [flags 0x400 0x0 ] (highWordZero Unsigned X!=0 X>=0 allocationCanBeRemoved )  [    0xffff08236e50] bci=[18,0,-] rc=6 vc=3599 vn=- li=45 udi=- nc=1 flg=0x4104
n2042n        loadaddr  net/openj9/test/binaryData/TestByteArray2Integer[#645  Static] [flags 0x18307 0x0 ]  [    0xffff08236e00] bci=[18,0,-] rc=1 vc=3599 vn=- li=45 udi=- nc=0
n2045n    allocationFence on n2043n (X==0 X>=0 X<=0 )                                         [    0xffff08236ef0] bci=[18,0,-] rc=0 vc=3599 vn=- li=45 udi=- nc=0 flg=0x302
n2049n    BBEnd </block_45>                                                                   [    0xffff08237030] bci=[18,13,-] rc=0 vc=3599 vn=- li=45 udi=- nc=0

aconst node (n2296n) says that class pointer of sun/reflect/GeneratedConstructorAccessor10 is 0x459000. However, 0x459000 is sun/reflect/GeneratedConstructorAccessor245 according to javacore. Possibly, sun/reflect/GeneratedConstructorAccessor10 has been already unloaded and sun/reflect/GeneratedConstructorAccessor245 has been loaded to the same address, and the class unload pic for this aconst node is missing.

2CLTEXTCLLOAD           Loader sun/reflect/DelegatingClassLoader(0x00000000F2091010)
3CLTEXTCLASS                    sun/reflect/GeneratedConstructorAccessor245(0x0000000000459000)
Akira1Saitoh commented 10 months ago

ifacmpne in the above trace is processed in ificmpHelper function, which generates a compare immediate instruction if the second child node is a constant value node and the value can be encoded into the instruction, rather than evaluating the second child node and generating a compare register instruction. I believe that the problem is that when it skips evaluating the second child node, it does not add class unload pic even if it is required. I am preparing the fix for it. https://github.com/eclipse/omr/blob/1b204d4703fe87dab6ee438bd49fdbff273cb551/compiler/aarch64/codegen/ControlFlowEvaluator.cpp#L229-L238

Akira1Saitoh commented 10 months ago

I opened https://github.com/eclipse/omr/pull/7206. 100x Grinder passed with this change. https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/36899/

knn-k commented 10 months ago

I reviewed and merged https://github.com/eclipse/omr/pull/7206 . I am going to run DaaLoadTest jobs when the fix becomes available in nightly builds.

pshipton commented 10 months ago

The OMR acceptance build is not good, there are crashes and assertions on aarch64.

https://openj9-jenkins.osuosl.org/job/Pipeline-OMR-Acceptance/621/ https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_sanity.functional_aarch64_linux_OMR_testList_0/57

OpenJ9 also changed since the last nighty build started, where aarch64 testing was fine. https://github.com/eclipse-openj9/openj9/compare/0ce57acd02b...52a09039db7

pshipton commented 10 months ago

Started new testing on the openj9 changes to confirm if they are good. https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-JDK21/177/

pshipton commented 10 months ago

Looks like the cause is the openj9 change, opened https://github.com/eclipse-openj9/openj9/issues/18626

pshipton commented 10 months ago

I promoted the OMR change, it will be in the nightly builds.

knn-k commented 10 months ago

I think we can close this issue now. I ran DaaLoadTest_all_special_5m_22 with Java 8 and 11, 80 times each, with nightly builds that contain the fix https://github.com/eclipse/omr/pull/7206. I got no failures.

Nightly builds:

Test jobs:

Thank you for your investigation and fix, @Akira1Saitoh !

knn-k commented 10 months ago

One more comment. The class data are allocated at addresses above the first 32-bit space on AArch64 macOS, and you cannot embed those addresses as immediate values in the compare instruction. That is why we have seen this failure on AArch64 Linux but not on macOS.