search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.27k stars 721 forks source link

HCRLateAttachWorkload_previewEnabled crash vmState=0x00050bad #17789

Open pshipton opened 1 year ago

pshipton commented 1 year ago

https://openj9-jenkins.osuosl.org/job/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/578/ HCRLateAttachWorkload_previewEnabled_1 -XX:-UseCompressedOops

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/578/system_test_output.tar.gz

LT  07:56:56.703 - Completed 10.1%. Number of tests started=2019 (+794)
LT  stderr #0: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0xa4e256) [0x3ffb5ece256]
LT  stderr #1: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0xa5bbf0) [0x3ffb5edbbf0]
LT  stderr #2: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x16e7d2) [0x3ffb55ee7d2]
LT  stderr #3: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9prt29.so(+0x3365c) [0x3ffb71b365c]
LT  stderr #4: [0x3ffb43f5230]
LT  stderr #5: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x26ff0a) [0x3ffb56eff0a]
LT  stderr #6: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x1549b2) [0x3ffb55d49b2]
LT  stderr #7: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x697028) [0x3ffb5b17028]
LT  stderr #8: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x698d7a) [0x3ffb5b18d7a]
LT  stderr #9: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x98a832) [0x3ffb5e0a832]
LT  stderr #10: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x98f0b0) [0x3ffb5e0f0b0]
LT  stderr #11: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x98fff6) [0x3ffb5e0fff6]
LT  stderr #12: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x8b423a) [0x3ffb5d3423a]
LT  stderr #13: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x8b6d5c) [0x3ffb5d36d5c]
LT  stderr #14: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x8b7436) [0x3ffb5d37436]
LT  stderr #15: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x674be2) [0x3ffb5af4be2]
LT  stderr #16: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x1871ae) [0x3ffb56071ae]
LT  stderr #17: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x188290) [0x3ffb5608290]
LT  stderr #18: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9prt29.so(+0x34546) [0x3ffb71b4546]
LT  stderr #19: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x18589a) [0x3ffb560589a]
LT  stderr #20: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x185f42) [0x3ffb5605f42]
LT  stderr #21: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x18494e) [0x3ffb560494e]
LT  stderr #22: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x184e3e) [0x3ffb5604e3e]
LT  stderr #23: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x184f22) [0x3ffb5604f22]
LT  stderr #24: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9prt29.so(+0x34546) [0x3ffb71b4546]
LT  stderr #25: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so(+0x1853be) [0x3ffb56053be]
LT  stderr #26: /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9thr29.so(+0x5a14) [0x3ffb7105a14]
LT  stderr #27: /lib/s390x-linux-gnu/libpthread.so.0(+0x7e66) [0x3ffb7d87e66]
LT  stderr #28: /lib/s390x-linux-gnu/libc.so.6(+0xfcbe6) [0x3ffb7b7cbe6]
LT  stderr #29: [(nil)]
LT  stderr Unhandled exception
LT  stderr Type=Segmentation error vmState=0x00050bad
LT  stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000002
LT  stderr Handler1=000003FFB72CD578 Handler2=000003FFB71B33D8 InaccessibleAddress=000003FEB539E000
LT  stderr gpr0=000003FF10A28D20 gpr1=000003FEB539EFD4 gpr2=000003FEB539EFD6 gpr3=000003FFB62F27D8
LT  stderr gpr4=000003FFB56EFEFC gpr5=000003FFB00EB7E0 gpr6=000003FF10850F50 gpr7=000003FF10801D98
LT  stderr gpr8=0000000000000000 gpr9=000003FFB0148000 gpr10=000003FE800CC900 gpr11=000003FFB43F5870
LT  stderr gpr12=000003FFB62F2000 gpr13=000003FFB43F5874 gpr14=000003FFB56EFEFC gpr15=000003FFB43F56B8
LT  stderr psw=000003FFB56EFF0A mask=0705000180000000 fpc=00880000 bea=000003FFB56A97C8
LT  stderr fpr0 40a4200000000000 (f: 0.000000, d: 2.576000e+03)
LT  stderr fpr1 000003ffb6124f0a (f: 3054653184.000000, d: 2.172311e-311)
LT  stderr fpr2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr3 bd7288dacb799331 (f: 3413742336.000000, d: -1.053569e-12)
LT  stderr fpr4 40a4200000000000 (f: 0.000000, d: 2.576000e+03)
LT  stderr fpr5 bfcef22dc8df3d72 (f: 3370073344.000000, d: -2.417657e-01)
LT  stderr fpr6 000003ffb43f5ee0 (f: 3024051968.000000, d: 2.172296e-311)
LT  stderr fpr7 3e1ae15e4ba2aa0b (f: 1268951552.000000, d: 1.564642e-09)
LT  stderr fpr8 0000000000101000 (f: 1052672.000000, d: 5.200891e-318)
LT  stderr fpr9 000002aa3083bc00 (f: 813939712.000000, d: 1.447603e-311)
LT  stderr fpr10 000003ffb42fe000 (f: 3023036416.000000, d: 2.172295e-311)
LT  stderr fpr11 000003ffc467bb20 (f: 3295132416.000000, d: 2.172430e-311)
LT  stderr fpr12 0000000000c7a458 (f: 13083736.000000, d: 6.464224e-317)
LT  stderr fpr13 000002aa2e9a2938 (f: 781855040.000000, d: 1.447587e-311)
LT  stderr fpr14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr Module=/home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so
LT  stderr Module_base_address=000003FFB5480000
LT  stderr 
LT  stderr Method_being_compiled=java/lang/String.<init>([C)V
LT  stderr Target=2_90_20230712_587 (Linux 5.4.0-137-generic)
LT  stderr CPU=s390x (4 logical CPUs) (0x1f58f7000 RAM)
LT  stderr ----------- Stack Backtrace -----------
LT  stderr _ZN11TR_J9VMBase17getClassNameCharsEP19TR_OpaqueClassBlockRi.localalias.211+0x62 (0x000003FFB56EFF0A [libj9jit29.so+0x26ff0a])
LT  stderr _ZN2J912AliasBuilder13methodAliasesEPN2TR15SymbolReferenceE+0x26a (0x000003FFB55D49B2 [libj9jit29.so+0x1549b2])
LT  stderr _ZN3OMR15SymbolReference18getUseDefAliasesBVEbb+0x470 (0x000003FFB5B17028 [libj9jit29.so+0x697028])
LT  stderr _ZN3OMR15SymbolReference12sharesSymbolEb+0xca (0x000003FFB5B18D7A [libj9jit29.so+0x698d7a])
LT  stderr _ZN13TR_UseDefInfo20indexSymbolsAndNodesERNS_13AuxiliaryDataE+0x3f2 (0x000003FFB5E0A832 [libj9jit29.so+0x98a832])
LT  stderr _ZN13TR_UseDefInfo17prepareUseDefInfoEbbbb+0x2f8 (0x000003FFB5E0F0B0 [libj9jit29.so+0x98f0b0])
LT  stderr _ZN13TR_UseDefInfoC1EPN2TR11CompilationEPNS0_3CFGEPNS0_9OptimizerEbbbbbbb+0x326 (0x000003FFB5E0FFF6 [libj9jit29.so+0x98fff6])
LT  stderr _ZN3OMR9Optimizer16createUseDefInfoEPN2TR11CompilationEbbbbbb+0xea (0x000003FFB5D3423A [libj9jit29.so+0x8b423a])
LT  stderr _ZN3OMR9Optimizer19performOptimizationEPK20OptimizationStrategyiii+0x1c0c (0x000003FFB5D36D5C [libj9jit29.so+0x8b6d5c])
LT  stderr _ZN3OMR9Optimizer8optimizeEv+0x1f6 (0x000003FFB5D37436 [libj9jit29.so+0x8b7436])
LT  stderr _ZN3OMR11Compilation7compileEv+0xbca (0x000003FFB5AF4BE2 [libj9jit29.so+0x674be2])
LT  stderr _ZN2TR28CompilationInfoPerThreadBase7compileEP10J9VMThreadPNS_11CompilationEP17TR_ResolvedMethodR11TR_J9VMBaseP19TR_OptimizationPlanRKNS_16SegmentAllocatorE+0x4c6 (0x000003FFB56071AE [libj9jit29.so+0x1871ae])
LT  stderr _ZN2TR28CompilationInfoPerThreadBase14wrappedCompileEP13J9PortLibraryPv+0x378 (0x000003FFB5608290 [libj9jit29.so+0x188290])
LT  stderr omrsig_protect+0x366 (0x000003FFB71B4546 [libj9prt29.so+0x34546])
LT  stderr _ZN2TR28CompilationInfoPerThreadBase7compileEP10J9VMThreadP21TR_MethodToBeCompiledRN2J917J9SegmentProviderE+0x332 (0x000003FFB560589A [libj9jit29.so+0x18589a])
LT  stderr _ZN2TR24CompilationInfoPerThread12processEntryER21TR_MethodToBeCompiledRN2J917J9SegmentProviderE+0x1d2 (0x000003FFB5605F42 [libj9jit29.so+0x185f42])
LT  stderr _ZN2TR24CompilationInfoPerThread14processEntriesEv+0x3c6 (0x000003FFB560494E [libj9jit29.so+0x18494e])
LT  stderr _ZN2TR24CompilationInfoPerThread3runEv+0x66 (0x000003FFB5604E3E [libj9jit29.so+0x184e3e])
LT  stderr _Z30protectedCompilationThreadProcP13J9PortLibraryPN2TR24CompilationInfoPerThreadE+0x9a (0x000003FFB5604F22 [libj9jit29.so+0x184f22])
LT  stderr omrsig_protect+0x366 (0x000003FFB71B4546 [libj9prt29.so+0x34546])
LT  stderr _Z21compilationThreadProcPv+0x1fe (0x000003FFB56053BE [libj9jit29.so+0x1853be])
LT  stderr thread_wrapper+0x114 (0x000003FFB7105A14 [libj9thr29.so+0x5a14])
LT  stderr start_thread+0xd6 (0x000003FFB7D87E66 [libpthread.so.0+0x7e66])
LT  stderr  (0x000003FFB7B7CBE6 [libc.so.6+0xfcbe6])
LT  stderr  (0x0000000000000000 [<unknown>+0x0])
pshipton commented 1 year ago

@r30shah

r30shah commented 1 year ago

Internally launched a grinder Test_grinder/job/Grinder/33958 to see if it fails frequently. It does not fail in 10 iterations. Initial inspection of the core-dump from the output files suggests we are failing in [1], looking at the parameters passed into this function the ramClass it does have valid value.

@Spencer-Comin Can you take a look at these failures?

[1]. https://github.com/eclipse-openj9/openj9/blob/f7305e7f338a61150e71235eea611d88f946b887/runtime/compiler/env/VMJ9.cpp#L2699

Spencer-Comin commented 1 year ago

Launched another grinder internally (Test_grinder/job/Grinder/34030). No failure in 50 iterations.

Spencer-Comin commented 1 year ago

Also ran the test locally on a development server, no failure in ~20 iterations.