search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.29k stars 722 forks source link

SC_Softmx_UpDown_0 crash vmState=0x00000000 #14129

Open pshipton opened 2 years ago

pshipton commented 2 years ago

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_extended.system_ppc64le_linux_Nightly_testList_1/166 - ub18-ppcle-2 SC_Softmx_UpDown_0 -Xjit -Xgcpolicy:gencon -Xnocompressedrefs

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk8_j9_extended.system_ppc64le_linux_Nightly_testList_1/166/system_test_output.tar.gz

STF 22:38:57.004 - +------ Step 9 - Wait for processes to complete

4.jvm1.stderr

Unhandled exception
Type=Segmentation error vmState=0x00000000
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=0000794FF40734B0 Handler2=0000794FEFF7C300
R0=FFFF86B133FF7A68 R1=0000794F36A7B230 R2=0000794FF4276E00 R3=0000794ECC008598
R4=0000794FF063B900 R5=0000000000000000 R6=0000794ECC008599 R7=0000794FF0027850
R8=0000000000000000 R9=0000000000000001 R10=00000000000001A0 R11=0000794ECC008598
R12=0000794FF5003B80 R13=0000794F36A868E0 R14=FFFFFFFFFFFFFFF8 R15=FFFFFFFFFFFFFF0F
R16=0000000000010000 R17=0000000000000000 R18=0000000000404000 R19=0000794F5420CCED
R20=0000000000000001 R21=0000000000000000 R22=0000794ECC008590 R23=0000000000000000
R24=0000794FF063B800 R25=0000000000000000 R26=FFFFFFFFFFFFFFFD R27=0000000000000001
R28=0000000000000000 R29=0000000000000000 R30=0000000000000000 R31=0000794FF045F260
NIP=0000794FF4101B3C MSR=800000010280F033 ORIG_GPR3=00000000000081C8 CTR=0000794FF4101AE0
LINK=0000794FF40FE654 XER=0000000020000000 CCR=0000000028004884 SOFTE=0000000000000001
TRAP=0000000000000300 DAR=00000000000001A0 dsisr=0000000040000000 RESULT=0000000000000000
Failed to find class net/adoptopenjdk/test/nio2/asyncio/client/MultipleReadWriteFutureTest in shared cache for class-loader id 0.
FPR0 0000794ecc0085f0 (f: 3422586368.000000, d: 6.589815e-310)
FPR1 4053572220000000 (f: 536870912.000000, d: 7.736146e+01)
FPR2 41e0000000000000 (f: 0.000000, d: 2.147484e+09)
FPR3 3fee666660000000 (f: 1610612736.000000, d: 9.500000e-01)
FPR4 4018c0f560000000 (f: 1610612736.000000, d: 6.188436e+00)
FPR5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR6 3fe62e42fefa39ef (f: 4277811712.000000, d: 6.931472e-01)
FPR7 3fd1058bc8a07ee1 (f: 3365961472.000000, d: 2.659635e-01)
FPR8 bfdffffef20a4123 (f: 4060758272.000000, d: -4.999997e-01)
FPR9 bfd00ea348b88334 (f: 1220051712.000000, d: -2.508934e-01)
FPR10 bfe0416d5ec8c504 (f: 1590215936.000000, d: -5.079867e-01)
FPR11 41cdcd6500000000 (f: 0.000000, d: 1.000000e+09)
FPR12 40eec30000000000 (f: 0.000000, d: 6.300000e+04)
FPR13 0000000000000001 (f: 1.000000, d: 4.940656e-324)
FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
Module=/home/jenkins/workspace/Test_openjdk8_j9_extended.system_ppc64le_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/ppc64le/default/libj9vm29.so
Module_base_address=0000794FF4030000
Target=2_90_20211209_173 (Linux 4.15.0-96-generic)
CPU=ppc64le (16 logical CPUs) (0x1fdc70000 RAM)
----------- Stack Backtrace -----------
(0x0000794FF4101B3C [libj9vm29.so+0xd1b3c])
(0x0000794FF41993C0 [libj9vm29.so+0x1693c0])
(0x0000794FF40492D0 [libj9vm29.so+0x192d0])
(0x0000794FF40D9ED0 [libj9vm29.so+0xa9ed0])
(0x0000794FEFF7D708 [libj9prt29.so+0x3d708])
(0x0000794FF40D5104 [libj9vm29.so+0xa5104])
(0x0000794FEFF113E8 [libj9thr29.so+0x113e8])
(0x0000794FF51D885C [libpthread.so.0+0x885c])
clone+0x98 (0x0000794FF5089028 [libc.so.6+0x159028])
---------------------------------------
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at java/lang/Class$CacheKey.<init>(Class.java:3813)
4XESTACKTRACE                at java/lang/Class$CacheKey.newMethodKey(Class.java:3797)
4XESTACKTRACE                at java/lang/Class.cacheMethods(Class.java:4197)
4XESTACKTRACE                at java/lang/Class.getDeclaredMethods(Class.java:1017)
4XESTACKTRACE                at org/junit/internal/MethodSorter.getDeclaredMethods(MethodSorter.java:54)
4XESTACKTRACE                at junit/framework/TestSuite.addTestsFromTestCase(TestSuite.java:149)
4XESTACKTRACE                at junit/framework/TestSuite.<init>(TestSuite.java:129)
4XESTACKTRACE                at org/junit/internal/runners/JUnit38ClassRunner.<init>(JUnit38ClassRunner.java:74)
pshipton commented 2 years ago

@tajila fyi

tajila commented 2 years ago

THe stackslots in the crashing thread:

<794ff08c5c00>  Bytecode index = 1
<794ff08c5c00>  Using local mapper
<794ff08c5c00>  Locals starting at 0x0000794EA40085C0 for 0x0000000000000009 slots
<794ff08c5c00>      O-Slot: a0[0x0000794EA40085C0] = 0x0000794FEDCF60D0
<794ff08c5c00>      O-Slot: a1[0x0000794EA40085B8] = 0x0000794F6FB05F68
<794ff08c5c00>      O-Slot: a2[0x0000794EA40085B0] = 0x0000794FED9FB9B0
<794ff08c5c00>      O-Slot: a3[0x0000794EA40085A8] = 0x0000794F6EC605E8
<794ff08c5c00>      I-Slot: t4[0x0000794EA40085A0] = 0x0000794FF0291520
<794ff08c5c00>      I-Slot: t5[0x0000794EA4008598] = 0x0000794FED9FB9B0
<794ff08c5c00>      I-Slot: t6[0x0000794EA4008590] = 0x0000794FED9F2ED0
<794ff08c5c00>      I-Slot: t7[0x0000794EA4008588] = 0x0000794EA40085C8
<794ff08c5c00>      I-Slot: t8[0x0000794EA4008580] = 0x0000794F5412FF0F
<794ff08c5c00>  Pending stack starting at 0x0000794EA4008560 for UDATA(0x0000000000000001) slots
<794ff08c5c00>      O-Slot: p0[0x0000794EA4008560] = 0x0000794F6FB05F68

The bytecodes of the method

> !bytecodes 0x0000794FF045F260
  Name: <init>
  Signature: (Ljava/lang/String;[Ljava/lang/Class;Ljava/lang/Class;)V
  Access Flags (13240002): private
  Max Stack: 2
  Argument Count: 4
  Temp Count: 5

    0 aload0
    1 invokespecial 12 java/lang/Object.<init>()V
    4 aload1
    5 invokevirtual 13 java/lang/String.hashCode()I
    8 aload3
    9 invokestatic 14 java/lang/Class$CacheKey.hashCombine(ILjava/lang/Object;)I

The first bytecode is an aload0 which should just load the first arg slot on the stack. But the stackslots show that they are different even though they should be the same.

<794ff08c5c00>      O-Slot: a0[0x0000794EA40085C0] = 0x0000794FEDCF60D0
...
<794ff08c5c00>  Pending stack starting at 0x0000794EA4008560 for UDATA(0x0000000000000001) slots
<794ff08c5c00>      O-Slot: p0[0x0000794EA4008560] = 0x0000794F6FB05F68

The pending push slot is a corrupt object (all zeroes) causing the crash.

tajila commented 2 years ago

The corrupt object from a method name in a java/lang/reflect/Method

> !j9object 0x0000794FED9F2ED0
!J9Object 0x0000794FED9F2ED0 {
    struct J9Class* clazz = !j9class 0x794FF0310C00 // java/lang/reflect/Method
    Object flags = 0x00000020;
    J lockword = 0x0000000000000008 (offset = 0) (java/lang/Object) <hidden>
    Z override = 0x00000000 (offset = 16) (java/lang/reflect/AccessibleObject)
    Ljava/lang/Object; securityCheckCache = !fj9object 0x0 (offset = 8) (java/lang/reflect/AccessibleObject)
    Z hasRealParameterData = 0x00000000 (offset = 20) (java/lang/reflect/Executable)
    [Ljava/lang/reflect/Parameter; parameters = !fj9object 0x0 (offset = 24) (java/lang/reflect/Executable)
    Ljava/util/Map; declaredAnnotations = !fj9object 0x0 (offset = 32) (java/lang/reflect/Executable)
    Ljava/lang/Class; clazz = !fj9object 0x794f6f138d58 (offset = 40) (java/lang/reflect/Method)
    I slot = 0x00000017 (offset = 136) (java/lang/reflect/Method)
    Ljava/lang/String; name = !fj9object 0x794f6fb05f68 (offset = 48) (java/lang/reflect/Method)//<--- corrupt object
    Ljava/lang/Class; returnType = !fj9object 0x794f6ec605e8 (offset = 56) (java/lang/reflect/Method)
    [Ljava/lang/Class; parameterTypes = !fj9object 0x794fed9fb9b0 (offset = 64) (java/lang/reflect/Method)
    [Ljava/lang/Class; exceptionTypes = !fj9object 0x794fed9fb9d8 (offset = 72) (java/lang/reflect/Method)
    I modifiers = 0x00000009 (offset = 140) (java/lang/reflect/Method)
    Ljava/lang/String; signature = !fj9object 0x0 (offset = 80) (java/lang/reflect/Method)
    Lsun/reflect/generics/repository/MethodRepository; genericInfo = !fj9object 0x0 (offset = 88) (java/lang/reflect/Method)
    [B annotations = !fj9object 0x0 (offset = 96) (java/lang/reflect/Method)
    [B parameterAnnotations = !fj9object 0x0 (offset = 104) (java/lang/reflect/Method)
    [B annotationDefault = !fj9object 0x0 (offset = 112) (java/lang/reflect/Method)
    Lsun/reflect/MethodAccessor; methodAccessor = !fj9object 0x0 (offset = 120) (java/lang/reflect/Method)
    Ljava/lang/reflect/Method; root = !fj9object 0x0 (offset = 128) (java/lang/reflect/Method)
}
tajila commented 2 years ago

The bad reference (0x794f6fb05f68) is within java heap range

+----------------+----------------+----------------+----------------+--------+----------------+----------------------
|    region      |     start      |      end       |    subspace    | flags  |      size      |      region type
+----------------+----------------+----------------+----------------+--------+----------------+----------------------
 0000794ff00f80f0 0000794f6ec60000 0000794f70c50000 0000794ff00e3ab0 00000009          1ff0000 ADDRESS_ORDERED
 0000794ff00f76a0 0000794fed9f0000 0000794fedeb0000 0000794ff00f0ec0 0000000a           4c0000 ADDRESS_ORDERED
 0000794ff00f6c50 0000794fedeb0000 0000794fee370000 0000794ff00ea3e0 0000000a           4c0000 ADDRESS_ORDERED
+----------------+----------------+----------------+----------------+--------+----------------+----------------------
tajila commented 2 years ago

@dmitripivkine Will the GC ever zero out ranges of the java heap? all the memory near 0x794f6fb05f68 seems to be zero

dmitripivkine commented 2 years ago

@dmitripivkine Will the GC ever zero out ranges of the java heap? all the memory near 0x794f6fb05f68 seems to be zero

There are multiple scenarios when GC might zero large portions of the heap, for instance:

An address 0x794f6fb05f68 is located in Tenure, so there is no TLH for Tenure in Gencon. Do you have core around? I can try to figure out why this location is zeroed. Most likely this object reference is stall and points mid another object

dmitripivkine commented 2 years ago

ah, I see link. I can download myself

dmitripivkine commented 2 years ago

There are a lot of large arrays around. This particular address is covered by this large Byte array (ends at 0x794F6FB22A48):

> !j9object 0x794F6FADAD50
!J9IndexableObject 0x0000794F6FADAD50 {
    struct J9Class* clazz = !j9arrayclass 0x794FF0293B00   // [B
    Object flags = 0x00000000;
    U_32 size = 0x00047CDE;
    [0] =   1, 0x01
    [1] =   1, 0x01
tajila commented 2 years ago

Okay thanks @dmitripivkine, looks like a stale pointer then.

tajila commented 2 years ago

30 job grinder, https://openj9-jenkins.osuosl.org/job/Grinder/474/

tajila commented 2 years ago

No failures in the 30 job grinder. Trying 100 https://openj9-jenkins.osuosl.org/job/Grinder/475/

tajila commented 2 years ago

No failures in the 100 job grinder.

pshipton commented 8 months ago

https://openj9-jenkins.osuosl.org/job/Test_openjdk11_j9_extended.system_aarch64_linux_Release_testList_1/88 - cent8-aarch64-6 SC_Softmx_UpDown_0

No diagnostic files.

Type=Segmentation error vmState=0x00000000
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=0000FFFF98701D94 Handler2=0000FFFF98667BB0 InaccessibleAddress=0000000000000020
R0=0000000000000008 R1=0000000000000018 R2=0000000000000012 R3=0000FFFF9402B1C0
R4=0000FFFF92883DC0 R5=0000FFFEE9813198 R6=000000000000049C R7=0000FFFF9402B1C0
R8=0000FFFF9439BD00 R9=0000FFFF1EA96850 R10=0000FFFE4403A6B0 R11=0000000000002400
R12=0000FFFE8F9ADFF0 R13=0000FFFE8F9ADAF0 R14=FFFFFFFFFFFFFB60 R15=0000000000000001
R16=0000000000000001 R17=0000FFFF98CF1D80 R18=0000FFFE8F9AE2F0 R19=0000FFFE4403AF00
R20=0000FFFE4403AB48 R21=0000FFFEE9813522 R22=0000000000001200 R23=0000000000000000
R24=0000FFFF988A62CC R25=0000000000000000 R26=0000FFFF92883DB0 R27=0000FFFE8F9AE020
R28=0000FFFF9876FC00 R29=0000FFFE8F9ADCB0 R30=0000FFFF98771790 R31=0000FFFE8F9ADCA0
PC=0000FFFF9876FF98 SP=0000FFFE8F9ADCA0 PSTATE=0000000060001000
V0 0000fffe4403ab90 (f: 1141091200.000000, d: 1.390634e-309)
V1 2f2f2f2f2f2f2f2f (f: 791621440.000000, d: 2.054695e-81)
V2 0000ffff24bedc81 (f: 616488064.000000, d: 1.390653e-309)
V3 62f9cd0ae27b6659 (f: 3799737856.000000, d: 6.085738e+168)
V4 d95af67c03e95457 (f: 65623128.000000, d: -2.784991e+122)
V5 000000005dc9a476 (f: 1573495936.000000, d: 7.774103e-315)
V6 94446f0194446f01 (f: 2487512832.000000, d: -4.855772e-211)
V7 000e1781000e1781 (f: 923521.000000, d: 1.959708e-308)
V8 000003c10000745f (f: 29791.000000, d: 2.039238e-311)
V9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V16 4010040140100401 (f: 1074791424.000000, d: 4.003911e+00)
V17 fcfdfe00000000ff (f: 255.000000, d: -1.197192e+294)
V18 8020080280200000 (f: 2149580800.000000, d: -4.458850e-308)
V19 3fc7466496cb03de (f: 2529887232.000000, d: 1.818357e-01)
V20 3fb1eb851eb851ec (f: 515396064.000000, d: 7.000000e-02)
V21 0000000000000008 (f: 8.000000, d: 3.952525e-323)
V22 3f0000003f800000 (f: 1065353216.000000, d: 3.051759e-05)
V23 3fc999999999999a (f: 2576980480.000000, d: 2.000000e-01)
V24 3fd6666666666666 (f: 1717986944.000000, d: 3.500000e-01)
V25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V26 0000000000000001 (f: 1.000000, d: 4.940656e-324)
V27 000000000000000a (f: 10.000000, d: 4.940656e-323)
V28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V31 7ff0000000000000 (f: 0.000000, d: inf)
Module=/home/jenkins/workspace/Test_openjdk11_j9_extended.system_aarch64_linux_Release_testList_1/jdkbinary/j2sdk-image/lib/default/libj9vm29.so
Module_base_address=0000FFFF986D0000
Target=2_90_20240330_113 (Linux 4.18.0-547.el8.aarch64)
CPU=aarch64 (8 logical CPUs) (0x1d9990000 RAM)
----------- Stack Backtrace -----------
Failed to find class sun/nio/ch/Net$1 in shared cache for class-loader id 0.
Failed to store class sun/nio/ch/Net$1 in shared cache for class-loader id 0 with URL /home/jenkins/workspace/Test_openjdk11_j9_extended.system_aarch64_linux_Release_testList_1/jdkbinary/j2sdk-image/lib/modules (index 0).
Failed to find class java/net/ProtocolFamily in shared cache for class-loader id 0.
Failed to store class java/net/ProtocolFamily in shared cache for class-loader id 0 with URL /home/jenkins/workspace/Test_openjdk11_j9_extended.system_aarch64_linux_Release_testList_1/jdkbinary/j2sdk-image/lib/modules (index 0).
Failed to find class sun/net/ext/ExtendedSocketOptions in shared cache for class-loader id 0.
Failed to store class sun/net/ext/ExtendedSocketOptions in shared cache for class-loader id 0 with URL /home/jenkins/workspace/Test_openjdk11_j9_extended.system_aarch64_linux_Release_testList_1/jdkbinary/j2sdk-image/lib/modules (index 0).
Failed to find class jdk/net/ExtendedSocketOptions in shared cache for class-loader id 0.
Failed to store class jdk/net/ExtendedSocketOptions in shared cache for class-loader id 0 with URL /home/jenkins/workspace/Test_openjdk11_j9_extended.system_aarch64_linux_Release_testList_1/jdkbinary/j2sdk-image/lib/modules (index 0).
bytecodeLoopFull+0x398 (0x0000FFFF9876FF98 [libj9vm29.so+0x9ff98])
c_cInterpreter+0x54 (0x0000FFFF9881BBFC [libj9vm29.so+0x14bbfc])
---------------------------------------
JVMDUMP039I Processing dump event "gpf", detail "" at 2024/03/30 22:21:56 - please wait.
Unhandled exception
Type=Segmentation error vmState=0x0002000f
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000002
Handler1=0000FFFF98701D94 Handler2=0000FFFF98667BB0 InaccessibleAddress=0000FFFE8FB4FFF8
R0=0000000000000000 R1=FFFFFFFFFFFF8764 R2=2000FFFE28007ECC R3=0000FFFE8FB8C4D8
R4=0000000000000000 R5=FFFFFFFFFFFFFFEF R6=0000FFFF9402B1C0 R7=0000FFFE8FB8C458
R8=000000000000005A R9=0000FFFE8FB8C4E8 R10=000000000000000F R11=0000FFFE8FB8C528
R12=0000FFFEE9813C70 R13=0000FFFEE9813C86 R14=0000FFFE8FB8C528 R15=0000FFFF988A9430
R16=0000FFFF988F0128 R17=0000FFFF98CF2C40 R18=0000000000000004 R19=0000000000000001
R20=0000FFFEE9813C86 R21=0000FFFE8FB8C4D0 R22=0000FFFEE9813C70 R23=FFFFFFFFFFFFFFC8
R24=0000000000000040 R25=0000FFFEE9813C88 R26=0000FFFE8FB8C510 R27=0000000000000015
R28=0000FFFE8FB8C4D0 R29=0000FFFE8FB8C3F0 R30=0000FFFF988345B0 R31=0000FFFE8FB8C3D0
PC=0000FFFF98834834 SP=0000FFFE8FB8C3D0 PSTATE=0000000000001000
V0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V1 00000000b72b4861 (f: 3073067008.000000, d: 1.518297e-314)
V2 0000ffff2fef5ad9 (f: 804215488.000000, d: 1.390654e-309)
V3 bfd00ea348b88334 (f: 1220051712.000000, d: -2.508934e-01)
V4 bfe005d3a0b72ff4 (f: 2696359936.000000, d: -5.007113e-01)
V5 3fc526e57720db08 (f: 1998641920.000000, d: 1.652495e-01)
V6 3fe62e42fefa39ef (f: 4277811712.000000, d: 6.931472e-01)
V7 8020080280200802 (f: 2149582848.000000, d: -4.458850e-308)
V8 000003c10000745f (f: 29791.000000, d: 2.039238e-311)
V9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V16 4010040140100401 (f: 1074791424.000000, d: 4.003911e+00)
V17 aaa0aa008000aaa8 (f: 2147527424.000000, d: -2.325052e-103)
V18 8020080200000000 (f: 0.000000, d: -4.458848e-308)
V19 3fc7466496cb03de (f: 2529887232.000000, d: 1.818357e-01)
V20 3fb1eb851eb851ec (f: 515396064.000000, d: 7.000000e-02)
V21 0000000000000008 (f: 8.000000, d: 3.952525e-323)
V22 3f0000003f800000 (f: 1065353216.000000, d: 3.051759e-05)
V23 3fc999999999999a (f: 2576980480.000000, d: 2.000000e-01)
V24 3fd6666666666666 (f: 1717986944.000000, d: 3.500000e-01)
V25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V26 0000000000000001 (f: 1.000000, d: 4.940656e-324)
V27 000000000000000a (f: 10.000000, d: 4.940656e-323)
V28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
V31 41dfffffffc00000 (f: 4290772992.000000, d: 2.147484e+09)
Module=/home/jenkins/workspace/Test_openjdk11_j9_extended.system_aarch64_linux_Release_testList_1/jdkbinary/j2sdk-image/lib/default/libj9vm29.so
Module_base_address=0000FFFF986D0000
Target=2_90_20240330_113 (Linux 4.18.0-547.el8.aarch64)
CPU=aarch64 (8 logical CPUs) (0x1d9990000 RAM)
----------- Stack Backtrace -----------
free(): invalid pointer