JIT crash DaaLoadTest vmState=0x00000000

[pshipton]

This is from the past weekend. Perhaps a dup of another issue but I can't tell. @0xdaryl

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_special.system_ppc64le_linux_Personal_testList_1/2 - ub16p8j94 DaaLoadTest_daa3_special_5m_16 -XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_special.system_ppc64le_linux_Personal_testList_1/2/system_test_output.tar.gz

DLT stderr Type=Segmentation error vmState=0x00000000
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
DLT stderr Handler1=00003FFF97AD4970 Handler2=00003FFF97A0C300
DLT stderr R0=00003FFF77878CE8 R1=00003FFF37EEB740 R2=00003FFF972CF300 R3=00000000005E6000
DLT stderr R4=FFFFFFFFFFFFFFFF R5=0000000081B6F280 R6=0000000000000010 R7=00000000000000E0
DLT stderr R8=0000000081666240 R9=0000000000000001 R10=FFFFFFFFFFFFFFFF R11=00003FFF96FED508
DLT stderr R12=000000000029DB00 R13=00003FFF37EF6900 R14=00000000005EE110 R15=00000000005E6000
DLT stderr R16=00003FFF74CB0038 R17=0000000081689878 R18=0000000081400780 R19=00000000819009A8
DLT stderr R20=0000000081443108 R21=0000000081B12600 R22=0000000081F19B00 R23=0000000081FF7A50
DLT stderr R24=0000000081FB50C8 R25=00000000816ADFE0 R26=00000000816ADFE0 R27=000000008180D728
DLT stderr R28=0000000081634840 R29=00000000816661D8 R30=0000000000683400 R31=0000000081B6F280
DLT stderr NIP=00003FFF96FD6A30 MSR=800000010280F033 ORIG_GPR3=00000000000081C8 CTR=00003FFF96FED508
DLT stderr LINK=00003FFF96FED51C XER=0000000020000000 CCR=0000000042004248 SOFTE=0000000000000001
DLT stderr TRAP=0000000000000300 DAR=0000000000000017 dsisr=0000000040000000 RESULT=0000000000000000
DLT stderr FPR0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR1 4052dcb020000000 (f: 536870912.000000, d: 7.544825e+01)
DLT stderr FPR2 c032f703035cfc16 (f: 56425496.000000, d: -1.896489e+01)
DLT stderr FPR3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR4 3f499a0000000000 (f: 0.000000, d: 7.812977e-04)
DLT stderr FPR5 bef8f000c0000000 (f: 3221225472.000000, d: -2.378226e-05)
DLT stderr FPR6 3f67721ca0000000 (f: 2684354560.000000, d: 2.862030e-03)
DLT stderr FPR7 3ee12da020000000 (f: 536870912.000000, d: 8.191216e-06)
DLT stderr FPR8 0072007200410065 (f: 4259941.000000, d: 1.602208e-306)
DLT stderr FPR9 006f005400790061 (f: 7929953.000000, d: 1.379603e-306)
DLT stderr FPR10 0061006d0053006f (f: 5439599.000000, d: 7.565991e-307)
DLT stderr FPR11 380b001b00000000 (f: 0.000000, d: 9.918385e-39)
DLT stderr FPR12 0000000000000001 (f: 1.000000, d: 4.940656e-324)
DLT stderr FPR13 bfb7440000000000 (f: 0.000000, d: -9.088135e-02)
DLT stderr FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk17_j9_special.system_ppc64le_linux_Personal_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so
DLT stderr Module_base_address=00003FFF96420000
DLT stderr Target=2_90_20210904_20 (Linux 4.4.0-173-generic)
DLT stderr CPU=ppc64le (16 logical CPUs) (0x1fe090000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr (0x00003FFF96FD6A30 [libj9jit29.so+0xbb6a30])
DLT stderr (0x00003FFF96FEC54C [libj9jit29.so+0xbcc54c])
DLT stderr (0x00003FFF97AA9450 [libj9vm29.so+0x19450])
DLT stderr (0x00003FFF97B44490 [libj9vm29.so+0xb4490])
DLT stderr (0x00003FFF97A0D708 [libj9prt29.so+0x3d708])
DLT stderr (0x00003FFF97B3F644 [libj9vm29.so+0xaf644])
DLT stderr (0x00003FFF979A13E8 [libj9thr29.so+0x113e8])
DLT stderr (0x00003FFF9CB58040 [libpthread.so.0+0x8040])
DLT stderr clone+0x98 (0x00003FFF9CA73BB0 [libc.so.6+0x123bb0])
DLT stderr ---------------------------------------

[0xdaryl]

@zl-wang : please assign for triage. Only observed on Power Linux.

[zl-wang]

@IBMJimmyk please take a look

[mnalam-p]

The corefile/javacore or jitdump is not available on the test output capture. So, I tried to reproduce the issue on a P8 machine with following options -

Java17 (Same version as the reported crash)
Linux PPC8 (ubuntu)
TEST_ITERATION=10
OPENJ9_JAVA_OPTIONS="-Xjit:count=0" (Also tried optlevel=<warm/hot>)

Failed to reproduce the test and the test passed. @pshipton Any suggestion on reproducing the failure?

[pshipton]

Just to try more iterations, which you can do on multiple machines in parallel. It may be more intermittent and not reproducible in 10 iterations.

[mnalam-p]

Using the JDK from https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_special.system_ppc64le_linux_Personal_testList_1/2 , I ran a grinder on 5 parallel machine, each having 30 iterations. The grinder passed. @pshipton Any suggestions on reproducing it?

[pshipton]

Nope. If it's not reproducible it's not a priority to fix.

[pshipton]

Haven't seen this again, closing.