search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.23k stars 713 forks source link

jdk_security4_0_FAILED sun/security/krb5/auto/HttpsCB.java Illegal instruction vmState=0x00040000 #19284

Open JasonFengJ9 opened 3 months ago

JasonFengJ9 commented 3 months ago

Failure link

From an internal build(rhel8le-rtp-rt7-1):

java version "11.0.23-beta" 2024-04-16
IBM Semeru Runtime Certified Edition 11.0.23+7-202404060107 (build 11.0.23-beta+7-202404060107)
Eclipse OpenJ9 VM 11.0.23+7-202404060107 (build master-d1a1a70e5, JRE 11 Linux ppc64le-64-Bit Compressed References 20240405_820 (JIT enabled, AOT enabled)
OpenJ9   - d1a1a70e5
OMR      - 8037ccd04
JCL      - 1b6c09aa97 based on jdk-11.0.23+7)

Rerun in Grinder - Change TARGET to run only the failed test targets.

Optional info

Failure output (captured from console output)

[2024-04-06T03:08:39.073Z] variation: Mode150
[2024-04-06T03:08:39.073Z] JVM_OPTIONS:  -XX:+UseCompressedOops -Xverbosegclog 

[2024-04-06T03:11:30.663Z] TEST: sun/security/krb5/auto/HttpsCB.java

[2024-04-06T03:11:30.666Z] STDERR:
[2024-04-06T03:11:30.666Z] Creating GSSCredential
[2024-04-06T03:11:30.666Z] Unhandled exception
[2024-04-06T03:11:30.666Z] Type=Illegal instruction vmState=0x00040000
[2024-04-06T03:11:30.666Z] J9Generic_Signal_Number=00000048 Signal_Number=00000004 Error_Value=00000000 Signal_Code=00000001
[2024-04-06T03:11:30.666Z] Handler1=00007FFF9A5AF4C0 Handler2=00007FFF9A4D9820
[2024-04-06T03:11:30.666Z] R0=00007FFF785AC530 R1=00007FFF4019AAF0 R2=00007FFF78767F00 R3=00000000FFEBF240
[2024-04-06T03:11:30.666Z] R4=00000000FFE526D5 R5=00000000000007D0 R6=00007FFEFC02A440 R7=00007FFEFC02A278
[2024-04-06T03:11:30.666Z] R8=00007FFEFC02A2B8 R9=00000000000001B0 R10=0000000000000002 R11=00007FFF7860A030
[2024-04-06T03:11:30.666Z] R12=0000000000004400 R13=00007FFF401A68E0 R14=00000000004FAFD0 R15=000000000048B100
[2024-04-06T03:11:30.666Z] R16=0000000000000000 R17=0000000000000010 R18=00000000000007D1 R19=00000000FFEBF240
[2024-04-06T03:11:30.666Z] R20=00000000004FAF98 R21=00007FFEFC02A2B8 R22=00007FFEFC02A440 R23=0000000100000000
[2024-04-06T03:11:30.666Z] R24=00000000FFE526D5 R25=00000000FFEBF240 R26=00000000000007D0 R27=0000000000000000
[2024-04-06T03:11:30.666Z] R28=0000000000000000 R29=0000000000000000 R30=000000000000007D R31=00007FFEFC02A278
[2024-04-06T03:11:30.666Z] NIP=00007FFF784D452C MSR=800000000288F033 ORIG_GPR3=00007FFF785AC52C CTR=00007FFF784DB260
[2024-04-06T03:11:30.666Z] LINK=00007FFF785AC530 XER=0000000020000000 CCR=0000000048224444 SOFTE=0000000000000001
[2024-04-06T03:11:30.666Z] TRAP=0000000000000700 DAR=00007FFF4144CB10 dsisr=0000000042000000 RESULT=0000000000000000
[2024-04-06T03:11:30.666Z] FPR0 0000000000000011 (f: 17.000000, d: 8.399116e-323)
[2024-04-06T03:11:30.666Z] FPR1 405433dc20000000 (f: 536870912.000000, d: 8.081031e+01)
[2024-04-06T03:11:30.666Z] FPR2 41e0000000000000 (f: 0.000000, d: 2.147484e+09)
[2024-04-06T03:11:30.666Z] FPR3 3fee666660000000 (f: 1610612736.000000, d: 9.500000e-01)
[2024-04-06T03:11:30.666Z] FPR4 3fce840b4ac4e4d2 (f: 1254417664.000000, d: 2.384047e-01)
[2024-04-06T03:11:30.666Z] FPR5 bfe7154748bef6c8 (f: 1220474624.000000, d: -7.213475e-01)
[2024-04-06T03:11:30.666Z] FPR6 3fe62e42fefa39ef (f: 4277811712.000000, d: 6.931472e-01)
[2024-04-06T03:11:30.666Z] FPR7 3fc526e57720db08 (f: 1998641920.000000, d: 1.652495e-01)
[2024-04-06T03:11:30.666Z] FPR8 38e06062e0000000 (f: 3758096384.000000, d: 9.856253e-35)
[2024-04-06T03:11:30.666Z] FPR9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR13 0000000000002200 (f: 8704.000000, d: 4.300347e-320)
[2024-04-06T03:11:30.666Z] FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2024-04-06T03:11:30.666Z] Module=/lib64/libcrypto.so.3
[2024-04-06T03:11:30.666Z] Module_base_address=00007FFF78260000
[2024-04-06T03:11:30.666Z] Target=2_90_20240405_820 (Linux 4.18.0-513.18.1.el8_9.ppc64le)
[2024-04-06T03:11:30.666Z] CPU=ppc64le (4 logical CPUs) (0x1dcc60000 RAM)
[2024-04-06T03:11:30.666Z] ----------- Stack Backtrace -----------
[2024-04-06T03:11:30.666Z]  (0x00007FFF784D452C [libcrypto.so.3+0x27452c])
[2024-04-06T03:11:30.666Z]  (0x00007FFF785AC530 [libcrypto.so.3+0x34c530])
[2024-04-06T03:11:30.666Z]  (0x00007FFF785ACA50 [libcrypto.so.3+0x34ca50])
[2024-04-06T03:11:30.666Z]  (0x00007FFF78608F94 [libcrypto.so.3+0x3a8f94])
[2024-04-06T03:11:30.667Z]  (0x00007FFF7860A064 [libcrypto.so.3+0x3aa064])
[2024-04-06T03:11:30.667Z] EVP_EncryptUpdate+0xac (0x00007FFF7847DB0C [libcrypto.so.3+0x21db0c])
[2024-04-06T03:11:30.667Z] EVP_CipherUpdate+0x24 (0x00007FFF7847E754 [libcrypto.so.3+0x21e754])
[2024-04-06T03:11:30.667Z] Java_jdk_crypto_jniprovider_NativeCrypto_GCMEncrypt+0x25c (0x00007FFF7A0D558C [libjncrypto.so+0x558c])
[2024-04-06T03:11:30.667Z]  (0x00007FFF9A7BC498 [libj9vm29.so+0x24c498])
[2024-04-06T03:11:30.667Z] ffi_call_int+0xd4 (0x00007FFF9A7BC164 [libj9vm29.so+0x24c164])
[2024-04-06T03:11:30.667Z] bytecodeLoopCompressed+0x11914 (0x00007FFF9A6374A4 [libj9vm29.so+0xc74a4])
[2024-04-06T03:11:30.667Z]  (0x00007FFF9A72AD60 [libj9vm29.so+0x1bad60])
[2024-04-06T03:11:30.667Z] runJavaThread+0x240 (0x00007FFF9A586F10 [libj9vm29.so+0x16f10])
[2024-04-06T03:11:30.667Z] javaProtectedThreadProc+0x148 (0x00007FFF9A61E778 [libj9vm29.so+0xae778])
[2024-04-06T03:11:30.667Z] omrsig_protect+0x3e4 (0x00007FFF9A4DACD4 [libj9prt29.so+0x3acd4])
[2024-04-06T03:11:30.667Z] javaThreadProc+0x60 (0x00007FFF9A61A0A0 [libj9vm29.so+0xaa0a0])
[2024-04-06T03:11:30.667Z] thread_wrapper+0x190 (0x00007FFF9A46D820 [libj9thr29.so+0xd820])
[2024-04-06T03:11:30.667Z] start_thread+0xf8 (0x00007FFF9B009718 [libpthread.so.0+0x9718])
[2024-04-06T03:11:30.667Z] clone+0x74 (0x00007FFF9AEAB778 [libc.so.6+0x13b778])
[2024-04-06T03:11:30.667Z] ---------------------------------------
[2024-04-06T03:11:30.667Z] JVMDUMP039I Processing dump event "gpf", detail "" at 2024/04/05 20:11:16 - please wait.

[2024-04-06T03:11:30.667Z] TEST RESULT: Failed. Unexpected exit from test [exit code: 255]
[2024-04-06T03:11:30.667Z] --------------------------------------------------
[2024-04-06T03:15:46.264Z] Test results: passed: 138; failed: 1
[2024-04-06T03:15:52.942Z] Report written to /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_testList_0/aqa-tests/TKG/output_17123689507735/jdk_security4_0/report/html/report.html
[2024-04-06T03:15:52.942Z] Results written to /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_testList_0/aqa-tests/TKG/output_17123689507735/jdk_security4_0/work
[2024-04-06T03:15:52.942Z] Error: Some tests failed or other problems occurred.
[2024-04-06T03:15:52.942Z] -----------------------------------
[2024-04-06T03:15:52.942Z] jdk_security4_0_FAILED

50x internal grinder - failed at rhel8le-svl-rt1-1, passed at ubu22lert-4, ubu20le-svl-rt4-1, rhel7le-rtp-rt6-1, and sles12le-rtp-rt3-1.

pshipton commented 3 months ago

The crash occurs in OpenSSL. I wonder if it's a machine setup issue since it occurs on specific machines.

OpenSSL is OpenSSL 1.1.1k FIPS 25 Mar 2021 Possibly a dup of https://github.ibm.com/runtimes/infrastructure/issues/9147

@jasonkatonica fyi

cjjdespres commented 2 months ago

I've seen this in the JITServer nightly tests as well, e.g. in https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_jit_Personal/1005/. I would have thought that it would have the same cause as https://github.com/eclipse-openj9/openj9/issues/19325, since that particular machine appears to be rhel8/power 8/ppc64le, but if it was running with OpenSSL 1.1.1k FIPS then maybe not? I do see that this failure occurred on the 16th, which was after the upgrade to OpenSSL 3.0.7 on rhel8 (https://github.ibm.com/runtimes/infrastructure/issues/9147#issuecomment-76212158).