search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.24k stars 713 forks source link

j9mm.479 ASSERTION FAILED openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env))) #18098

Open JasonFengJ9 opened 10 months ago

JasonFengJ9 commented 10 months ago

Failure link

JDK8 s390x_linux 0.41 milestone 1(rhel7s390x-svl-rt10-1)

15:33:24  openjdk version "1.8.0_392"
15:33:24  IBM Semeru Runtime Open Edition (build 1.8.0_392-b03)
15:33:24  Eclipse OpenJ9 VM (build v0.41.0-release-87d042a68, JRE 1.8.0 Linux s390x-64-Bit Compressed References 20230906_725 (JIT enabled, AOT enabled)
15:33:24  OpenJ9   - 87d042a68
15:33:24  OMR      - fa7b6ddc7
15:33:24  JCL      - 04ddaa8f70 based on jdk8u392-b03)

Rerun in Grinder - Change TARGET to run only the failed test targets.

Optional info

Failure output (captured from console output)

[2023-09-06T20:06:36.004Z] variation: Mode650
[2023-09-06T20:06:36.004Z] JVM_OPTIONS:  -XX:-UseCompressedOops 

[2023-09-06T20:08:13.878Z] LT  stderr 20:08:12.044 0x3fea0084900    j9mm.479    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
[2023-09-06T20:08:13.878Z] LT  stderr 20:08:12.044 0x3fec4002200    j9mm.479    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
[2023-09-06T20:08:13.878Z] LT  stderr 000003FF9C830D00: framesWalked=3
[2023-09-06T20:08:13.878Z] LT  stderr 000003FF9C830D00: arg0EA=000003FEA0075FA8
[2023-09-06T20:08:13.878Z] LT  stderr 000003FF9C830D00: walkSP=000003FEA0075F80
[2023-09-06T20:08:13.878Z] LT  stderr 000003FF9C830D00: literals=000003FF9C59E0E8
[2023-09-06T20:08:13.878Z] LT  stderr 000003FF9C830D00: jitInfo=0000000000000000
[2023-09-06T20:08:13.878Z] LT  stderr 000003FF9C830D00: method=000003FF9C59E0E8 (net/adoptopenjdk/test/hcrAgent/agent/TransformerMakerThread.run()V) (Interpreted)
[2023-09-06T20:08:13.878Z] LT  stderr 000003FF9C830D00: stack=000003FEA0074820-000003FEA0076000
[2023-09-06T20:08:13.878Z] LT  stderr 20:08:12.044 0x3febc002200    j9mm.479    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))

[2023-09-06T20:08:34.384Z] STF 13:08:34.082 - Overall result: **FAILED**
[2023-09-06T20:08:34.384Z] -----------------------------------
[2023-09-06T20:08:34.384Z] HCRLateAttachWorkload_1_FAILED

50x internal grinder - 5/50 failed

dmitripivkine commented 10 months ago

@JasonFengJ9 @pshipton This failure looks like similar to https://github.com/eclipse-openj9/openj9/issues/13504 too. This one might be a duplicate.

JasonFengJ9 commented 10 months ago

There were 5 failures in the grinder. fyrlx11l

21:30:24  LT  stderr Unhandled exception
21:30:24  LT  stderr Type=Segmentation error vmState=0x00000000
21:30:24  LT  stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
21:30:24  LT  stderr Handler1=000003FFAB3C68B0 Handler2=000003FFAB2B3370 InaccessibleAddress=0000000000000000
21:30:24  LT  stderr gpr0=0000000000000000 gpr1=000003FFAB5E16B0 gpr2=000003FF181387F4 gpr3=00000000000000FF
21:30:24  LT  stderr gpr4=0000000000000040 gpr5=000003FE8C063EC0 gpr6=0000000000000000 gpr7=000003FF15877308
21:30:24  LT  stderr gpr8=000000000095A800 gpr9=000003FE84013D38 gpr10=000003FE8C064CB0 gpr11=000003FE8C054BE1
21:30:24  LT  stderr gpr12=000003FF00000030 gpr13=000003FFAB5A60C0 gpr14=000003FFAB440BB6 gpr15=000003FF15876E48
21:30:24  LT  stderr psw=000003FFAB441EDC mask=0705200180000000 fpc=00080000 bea=000003FFAB44954C
21:30:24  LT  stderr Unhandled exception
21:30:24  LT  stderr Type=Segmentation error vmState=0x00000000
21:30:24  LT  stderr fpr0 000003fe840006b0 (f: 2214594304.000000, d: 2.169774e-311)
21:30:24  LT  stderr fpr1 0000000000000007 (f: 7.000000, d: 3.458460e-323)
21:30:24  LT  stderr fpr2 000003fe840aef20 (f: 2215309056.000000, d: 2.169774e-311)
21:30:24  LT  stderr fpr3 000003fe84000f98 (f: 2214596608.000000, d: 2.169774e-311)
21:30:24  LT  stderr fpr4 000003fe84000a8e (f: 2214595328.000000, d: 2.169774e-311)
21:30:24  LT  stderr fpr5 35b818b6de54ffaf (f: 3730112512.000000, d: 6.440438e-50)
21:30:24  LT  stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
21:30:24  LT  stderr fpr7 000003ffd8a7d11c (f: 3634876672.000000, d: 2.172598e-311)
21:30:24  LT  stderr fpr8 000003ff15838000 (f: 360939520.000000, d: 2.170980e-311)
21:30:24  LT  stderr fpr9 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
21:30:24  LT  stderr fpr10 000003ffa47b47c0 (f: 2759542784.000000, d: 2.172165e-311)
21:30:24  LT  stderr fpr11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
21:30:24  LT  stderr fpr12 000003ff15878900 (f: 361203968.000000, d: 2.170980e-311)
21:30:24  LT  stderr fpr13 0000000000000001 (f: 1.000000, d: 4.940656e-324)
21:30:24  LT  stderr fpr14 000003ff15838000 (f: 360939520.000000, d: 2.170980e-311)
21:30:24  LT  stderr fpr15 0000000002727600 (f: 41055744.000000, d: 2.028423e-316)
21:30:24  LT  stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
21:30:24  LT  stderr Handler1=000003FFAB3C68B0 Handler2=000003FFAB2B3370 InaccessibleAddress=0000000000000000
21:30:24  LT  stderr Module=/home/jenkins/workspace/Grinder_iteration_0/openjdkbinary/j2sdk-image/jre/lib/s390x/default/libj9vm29.so
21:30:24  LT  stderr Module_base_address=000003FFAB380000
21:30:24  LT  stderr gpr0=0000000000000000 gpr1=000003FFAB5E16B0 gpr2=000003FF181387F4 gpr3=00000000000000FF
21:30:24  LT  stderr gpr4=0000000000000040 gpr5=000003FE8C063EC0 gpr6=0000000000000000 gpr7=000003FF1594D308
21:30:24  LT  stderr gpr8=0000000000954D00 gpr9=000003FE8400B358 gpr10=000003FE8C064CB0 gpr11=000003FE8C054BE1
21:30:24  LT  stderr gpr12=000003FF00000030 gpr13=000003FFAB5A60C0 gpr14=000003FFAB440BB6 gpr15=000003FF1594CE48
21:30:24  LT  stderr Target=2_90_20230906_725 (Linux 5.15.0-78-generic)
21:30:24  LT  stderr CPU=s390x (4 logical CPUs) (0x1e8fc7000 RAM)
21:30:24  LT  stderr psw=000003FFAB441EDC mask=0705200180000000 fpc=00080000 bea=000003FFAB44954C
21:30:24  LT  stderr ----------- Stack Backtrace -----------
21:30:24  LT  stderr fpr0 3ec13c7800000000 (f: 0.000000, d: 2.054716e-06)
21:30:24  LT  stderr fpr1 4844d24000000000 (f: 0.000000, d: 1.417024e+40)
21:30:24  LT  stderr fpr2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
21:30:24  LT  stderr fpr3 c036d171fe99c5fb (f: 4271490560.000000, d: -2.281815e+01)
21:30:24  LT  stderr fpr4 4844d24080000000 (f: 2147483648.000000, d: 1.417024e+40)
21:30:24  LT  stderr fpr5 000003ffab6cbe58 (f: 2876030464.000000, d: 2.172223e-311)
21:30:24  LT  stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
21:30:24  LT  stderr fpr7 000003ffd8a7d11c (f: 3634876672.000000, d: 2.172598e-311)
21:30:24  LT  stderr fpr8 000003ff1590e000 (f: 361816064.000000, d: 2.170980e-311)
21:30:24  LT  stderr fpr9 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
21:30:24  LT  stderr fpr10 000003fec00924f8 (f: 3221824768.000000, d: 2.170271e-311)
21:30:24  LT  stderr fpr11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
21:30:24  LT  stderr fpr12 000003ff1594e900 (f: 362080512.000000, d: 2.170981e-311)
21:30:24  LT  stderr fpr13 0000000000000001 (f: 1.000000, d: 4.940656e-324)
21:30:24  LT  stderr fpr14 000003ff1590e000 (f: 361816064.000000, d: 2.170980e-311)
21:30:24  LT  stderr fpr15 0000000002727600 (f: 41055744.000000, d: 2.028423e-316)
21:30:24  LT  stderr Module=/home/jenkins/workspace/Grinder_iteration_0/openjdkbinary/j2sdk-image/jre/lib/s390x/default/libj9vm29.so
21:30:24  LT  stderr Module_base_address=000003FFAB380000
21:30:24  LT  stderr Target=2_90_20230906_725 (Linux 5.15.0-78-generic)
21:30:24  LT  stderr CPU=s390x (4 logical CPUs) (0x1e8fc7000 RAM)
21:30:24  LT  stderr ----------- Stack Backtrace -----------
21:30:24  LT  stderr _ZN26VM_BytecodeInterpreterFull3runEP10J9VMThread+0x1314 (0x000003FFAB441EDC [libj9vm29.so+0xc1edc])
21:30:24  LT  stderr bytecodeLoopFull+0xf6 (0x000003FFAB440BB6 [libj9vm29.so+0xc0bb6])
21:30:24  LT  stderr c_cInterpreter+0x64 (0x000003FFAB5029A4 [libj9vm29.so+0x1829a4])
21:30:24  LT  stderr ---------------------------------------
21:30:24  LT  stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2023/09/08 01:30:23 - please wait.
21:30:24  LT  stderr _ZN26VM_BytecodeInterpreterFull3runEP10J9VMThread+0x1314 (0x000003FFAB441EDC [libj9vm29.so+0xc1edc])
21:30:24  LT  stderr bytecodeLoopFull+0xf6 (0x000003FFAB440BB6 [libj9vm29.so+0xc0bb6])
21:30:24  LT  stderr c_cInterpreter+0x64 (0x000003FFAB5029A4 [libj9vm29.so+0x1829a4])
21:30:24  LT  stderr ---------------------------------------
21:30:24  LT  stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2023/09/08 01:30:23 - please wait.

ubu22s390x-rt5-1

21:43:35  LT  stderr 01:43:34.098 0x3fed8002400    j9mm.479    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))

rhel9s390x-svl-rt1-1

21:38:09  LT  stderr 01:38:08.059 0x3fea8078100    j9mm.479    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
21:51:18  LT  stderr Unhandled exception
21:51:18  LT  stderr Type=Segmentation error vmState=0x0002000f
21:51:18  LT  stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
21:51:18  LT  stderr Handler1=000003FF8B746C40 Handler2=000003FF8B9B3370 InaccessibleAddress=0000000000000000
21:51:18  LT  stderr gpr0=000000000000009E gpr1=0000000000000000 gpr2=0000000099669966 gpr3=000003FF8A18DC26
21:51:18  LT  stderr gpr4=000003FF8B57E238 gpr5=000003FE90062780 gpr6=000003FE90062780 gpr7=000003FE90062780
21:51:18  LT  stderr gpr8=000003FF8B57E238 gpr9=000003FF8A204880 gpr10=0000000000000002 gpr11=000003FEA0004318
21:51:18  LT  stderr gpr12=000003FF8C3AEF68 gpr13=000003FF8BEC4008 gpr14=000003FF89F4A970 gpr15=000003FF8B57DDF0
21:51:18  LT  stderr psw=000003FF8A0B4702 mask=0705000180000000 fpc=00080000 bea=000003FF8A0B4660
21:51:18  LT  stderr fpr0 42a5526e08eeccc1 (f: 149867712.000000, d: 1.172189e+13)
21:51:18  LT  stderr fpr1 4036000000000000 (f: 0.000000, d: 2.200000e+01)
21:51:18  LT  stderr fpr2 3e3ab283fc9de728 (f: 4238206720.000000, d: 6.215956e-09)
21:51:18  LT  stderr fpr3 000003fea0001160 (f: 2684358912.000000, d: 2.170006e-311)
21:51:18  LT  stderr fpr4 bfe00a6cde5f8427 (f: 3730801664.000000, d: -5.012726e-01)
21:51:18  LT  stderr fpr5 230be88c2abd38b4 (f: 717043904.000000, d: 7.323637e-140)
21:51:18  LT  stderr fpr6 402e263ee26d0dcb (f: 3798797824.000000, d: 1.507470e+01)
21:51:18  LT  stderr fpr7 0000000000001000 (f: 4096.000000, d: 2.023693e-320)
21:51:18  LT  stderr fpr8 000003ff8b53f000 (f: 2337533952.000000, d: 2.171957e-311)
21:51:18  LT  stderr fpr9 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
21:51:18  LT  stderr fpr10 000003ff84221528 (f: 2216826112.000000, d: 2.171897e-311)
21:51:18  LT  stderr fpr11 000003ffd6078832 (f: 3590817792.000000, d: 2.172576e-311)
21:51:18  LT  stderr fpr12 000003ff8b57f900 (f: 2337798400.000000, d: 2.171957e-311)
21:51:18  LT  stderr fpr13 0000000000000001 (f: 1.000000, d: 4.940656e-324)
21:51:18  LT  stderr fpr14 000003ff8b53f000 (f: 2337533952.000000, d: 2.171957e-311)
21:51:18  LT  stderr fpr15 0000000000fb0ae0 (f: 16452320.000000, d: 8.128526e-317)
21:51:18  LT  stderr Module=/home/jenkins/workspace/Grinder_iteration_4/openjdkbinary/j2sdk-image/jre/lib/s390x/default/libj9gc_full29.so
21:51:18  LT  stderr Module_base_address=000003FF89F00000
21:51:18  LT  stderr Target=2_90_20230906_725 (Linux 5.14.0-284.25.1.el9_2.s390x)
21:51:18  LT  stderr CPU=s390x (4 logical CPUs) (0x1e5d62000 RAM)
21:51:18  LT  stderr ----------- Stack Backtrace -----------
21:51:18  LT  stderr _ZN23MM_ScavengerRootScanner11doStackSlotEPP8J9ObjectPvPKv+0xf2 (0x000003FF8A0B4702 [libj9gc_full29.so+0x1b4702])
21:51:18  LT  stderr vmThreadStackDoOSlotIterator+0x30 (0x000003FF89F4A970 [libj9gc_full29.so+0x4a970])
21:51:18  LT  stderr walkBytecodeFrameSlots+0x246 (0x000003FF8B786FEE [libj9vm29.so+0x86fee])
21:51:18  LT  stderr walkStackFrames+0x6cc (0x000003FF8B787B8C [libj9vm29.so+0x87b8c])
21:51:18  LT  stderr _ZN28GC_VMThreadStackSlotIterator9scanSlotsEP10J9VMThreadS1_PvPFvP8J9JavaVMPP8J9ObjectS2_P16J9StackWalkStatePKvEbb+0x4e (0x000003FF89F4AAA6 [libj9gc_full29.so+0x4aaa6])
21:51:18  LT  stderr _ZN14MM_RootScanner13scanOneThreadEP18MM_EnvironmentBaseP10J9VMThreadPv+0x14a (0x000003FF89F415DA [libj9gc_full29.so+0x415da])
21:51:18  LT  stderr _ZN14MM_RootScanner11scanThreadsEP18MM_EnvironmentBase+0xe2 (0x000003FF89F3FF5A [libj9gc_full29.so+0x3ff5a])
21:51:18  LT  stderr _ZN14MM_RootScanner9scanRootsEP18MM_EnvironmentBase+0x64 (0x000003FF89F4357C [libj9gc_full29.so+0x4357c])
21:51:18  LT  stderr _ZN12MM_Scavenger24workThreadGarbageCollectEP22MM_EnvironmentStandard+0x34e (0x000003FF8A0ADFEE [libj9gc_full29.so+0x1adfee])
21:51:18  LT  stderr _ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+0x218 (0x000003FF8A050AB8 [libj9gc_full29.so+0x150ab8])
21:51:18  LT  stderr _Z23dispatcher_thread_proc2P14OMRPortLibraryPv+0x138 (0x000003FF8A04F438 [libj9gc_full29.so+0x14f438])
21:51:18  LT  stderr omrsig_protect+0x366 (0x000003FF8B9B44DE [libj9prt29.so+0x344de])
21:51:18  LT  stderr dispatcher_thread_proc+0x60 (0x000003FF8A04F0B0 [libj9gc_full29.so+0x14f0b0])
21:51:18  LT  stderr thread_wrapper+0x114 (0x000003FF8B685A14 [libj9thr29.so+0x5a14])
21:51:18  LT  stderr start_thread+0x146 (0x000003FF8BF9E2F6 [libc.so.6+0x9e2f6])
21:51:18  LT  stderr  (0x000003FF8BF2FDBE [libc.so.6+0x2fdbe])
21:51:18  LT  stderr ---------------------------------------
21:51:18  LT  stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2023/09/07 18:51:17 - please wait.
22:05:43  LT  stderr Unhandled exception
22:05:43  LT  stderr Type=Segmentation error vmState=0x00000000
22:05:43  LT  stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
22:05:43  LT  stderr Handler1=000003FF87A468B0 Handler2=000003FF87933370 InaccessibleAddress=E350D0200024E000
22:05:43  LT  stderr gpr0=000003FEED1175A0 gpr1=E350D0200024E320 gpr2=000003FE8C08FD00 gpr3=000003FF885AE800
22:05:43  LT  stderr gpr4=000003FEED1175B8 gpr5=0000000000000008 gpr6=0000000000000000 gpr7=000003FE8C07F700
22:05:43  LT  stderr gpr8=000003FFD1059C2B gpr9=000003FF00000001 gpr10=0000000000000018 gpr11=000003FE8C07F700
22:05:43  LT  stderr gpr12=0000000000000000 gpr13=000003FE8C08FD00 gpr14=000003FF79C8B66E gpr15=000003FEEC067000
22:05:43  LT  stderr psw=000003FF79C781E4 mask=0705100180000000 fpc=00080000 bea=000003FF79C781EA
22:05:43  LT  stderr fpr0 3f06b23000000000 (f: 0.000000, d: 4.328927e-05)
22:05:43  LT  stderr fpr1 480587c000000000 (f: 0.000000, d: 9.157966e+38)
22:05:43  LT  stderr fpr2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
22:05:43  LT  stderr fpr3 000003fe600ebe68 (f: 1611579008.000000, d: 2.169476e-311)
22:05:43  LT  stderr fpr4 480587c080000000 (f: 2147483648.000000, d: 9.157969e+38)
22:05:43  LT  stderr fpr5 7a3bc61c7c810938 (f: 2088831232.000000, d: 6.301913e+280)
22:05:43  LT  stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
22:05:43  LT  stderr fpr7 0000000000001000 (f: 4096.000000, d: 2.023693e-320)
22:05:43  LT  stderr fpr8 000003feec029000 (f: 3959590912.000000, d: 2.170636e-311)
22:05:43  LT  stderr fpr9 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
22:05:43  LT  stderr fpr10 000003fe8c068c00 (f: 2349239296.000000, d: 2.169840e-311)
22:05:43  LT  stderr fpr11 000003ffea578832 (f: 3931604992.000000, d: 2.172744e-311)
22:05:43  LT  stderr fpr12 000003feec069900 (f: 3959855360.000000, d: 2.170636e-311)
22:05:43  LT  stderr fpr13 0000000000000001 (f: 1.000000, d: 4.940656e-324)
22:05:43  LT  stderr fpr14 000003feec029000 (f: 3959590912.000000, d: 2.170636e-311)
22:05:43  LT  stderr fpr15 0000000000fb0ae0 (f: 16452320.000000, d: 8.128526e-317)
22:05:43  LT  stderr Module=/home/jenkins/workspace/Grinder_iteration_4/openjdkbinary/j2sdk-image/jre/lib/s390x/default/libj9jit29.so
22:05:43  LT  stderr Module_base_address=000003FF79100000
22:05:43  LT  stderr Target=2_90_20230906_725 (Linux 5.14.0-284.25.1.el9_2.s390x)
22:05:43  LT  stderr CPU=s390x (4 logical CPUs) (0x1e5d62000 RAM)
22:05:43  LT  stderr ----------- Stack Backtrace -----------
22:05:43  LT  stderr old_fast_jitLookupInterfaceMethod+0xa4 (0x000003FF79C781E4 [libj9jit29.so+0xb781e4])
22:05:43  LT  stderr jitLookupInterfaceMethod+0x6e (0x000003FF79C8B66E [libj9jit29.so+0xb8b66e])
22:05:43  LT  stderr ---------------------------------------
22:05:43  LT  stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2023/09/07 19:05:42 - please wait.
22:05:43  LT  stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Grinder_iteration_4/aqa-tests/TKG/output_16941385334771/HCRLateAttachWorkload_1/20230907-190215-HCRLateAttachWorkload/results/core.20230907.190542.944064.0001.dmp' in response to an event
22:05:50  STF 19:05:47.850 - Found dump at: /home/jenkins/workspace/Grinder_iteration_4/aqa-tests/TKG/output_16941385334771/HCRLateAttachWorkload_1/20230907-190215-HCRLateAttachWorkload/results/core.20230907.190542.944064.0001.dmp
22:05:50  LT  stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Grinder_iteration_4/aqa-tests/TKG/output_16941385334771/HCRLateAttachWorkload_1/20230907-190215-HCRLateAttachWorkload/results/core.20230907.190542.944064.0001.dmp
22:05:50  LT  stderr 000003FE8C078100: Invalid class pointer in stack allocated object in thread Thread-330
22:05:50  LT  stderr 000003FE8C078100:  O-Slot=000003FE8C088080
22:05:50  LT  stderr 000003FE8C078100:  O-Slot value=000003FE8C087E88
22:05:50  LT  stderr 000003FE8C078100:  PC=000003FF885957C3
22:05:50  LT  stderr 000003FE8C078100:  framesWalked=3
22:05:50  LT  stderr 000003FE8C078100:  arg0EA=000003FE8C088088
22:05:50  LT  stderr 000003FE8C078100:  walkSP=000003FE8C088060
22:05:50  LT  stderr 000003FE8C078100:  literals=000003FF885B7368
22:05:50  LT  stderr 000003FE8C078100:  jitInfo=0000000000000000
22:05:50  LT  stderr 000003FE8C078100:  method=000003FF885B7368 (net/adoptopenjdk/test/hcrAgent/agent/TransformerMakerThread.run()V) (Interpreted)
22:05:50  LT  stderr 000003FE8C078100:  stack=000003FE8C086900-000003FE8C0880E0
22:05:50  LT  stderr 000003FE8C059F00: Invalid class pointer in stack allocated object in thread Thread-332
22:05:50  LT  stderr 000003FE8C059F00:  O-Slot=000003FE8C07F940
22:05:50  LT  stderr 000003FE8C059F00:  O-Slot value=000003FE8C07F748
22:05:50  LT  stderr 000003FE8C059F00:  PC=000003FF885957C3
22:05:50  LT  stderr 000003FE8C059F00:  framesWalked=3
22:05:50  LT  stderr 000003FE8C059F00:  arg0EA=000003FE8C07F948
22:05:50  LT  stderr 000003FE8C059F00:  walkSP=000003FE8C07F920
22:05:50  LT  stderr 000003FE8C059F00:  literals=000003FF885B7368
22:05:50  LT  stderr 000003FE8C059F00:  jitInfo=0000000000000000
22:05:50  LT  stderr 000003FE8C059F00:  method=000003FF885B7368 (net/adoptopenjdk/test/hcrAgent/agent/TransformerMakerThread.run()V) (Interpreted)
22:05:50  LT  stderr 000003FE8C059F00:  stack=000003FE8C07E1C0-000003FE8C07F9A0
22:05:50  LT  stderr Unhandled exception
22:05:50  LT  stderr Type=Segmentation error vmState=0x0002000f
22:05:50  LT  stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000002
22:05:50  LT  stderr Handler1=000003FF87A46C40 Handler2=000003FF87933370 InaccessibleAddress=000003FE5F448000
22:05:50  LT  stderr gpr0=0000000000000004 gpr1=000003FF78EB4610 gpr2=000003FF88061C20 gpr3=000003FE5F448758
22:05:50  LT  stderr gpr4=000003FF8C0FE238 gpr5=000003FE5F448758 gpr6=000003FE5F448758 gpr7=000003FF8C745034
22:05:50  LT  stderr gpr8=000003FE5F448750 gpr9=000000000000001F gpr10=000003FF5BB2A190 gpr11=000003FF8C0FE818
22:05:50  LT  stderr gpr12=000003FF8C7AEF68 gpr13=000003FF8C2C4008 gpr14=000003FF78D4A970 gpr15=000003FF8C0FDED0
22:05:50  LT  stderr psw=000003FF78EB4620 mask=0705100180000000 fpc=00080000 bea=000003FF78D41BB0
22:05:50  LT  stderr fpr0 000003ff8c4cdb10 (f: 2353847040.000000, d: 2.171965e-311)
22:05:50  LT  stderr fpr1 0000000000000007 (f: 7.000000, d: 3.458460e-323)
22:05:50  LT  stderr fpr2 000003ff8c4cdb10 (f: 2353847040.000000, d: 2.171965e-311)
22:05:50  LT  stderr fpr3 100003fea00008e8 (f: 2684356864.000000, d: 1.289486e-231)
22:05:50  LT  stderr fpr4 040003fea00008d6 (f: 2684356864.000000, d: 2.054270e-289)
22:05:50  LT  stderr fpr5 7a3bc61c7c810938 (f: 2088831232.000000, d: 6.301913e+280)
22:05:50  LT  stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
22:05:50  LT  stderr fpr7 0000000000001000 (f: 4096.000000, d: 2.023693e-320)
22:05:50  LT  stderr fpr8 000003ff8c0bf000 (f: 2349592576.000000, d: 2.171963e-311)
22:05:50  LT  stderr fpr9 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
22:05:50  LT  stderr fpr10 000003ff88373550 (f: 2285319424.000000, d: 2.171931e-311)
22:05:50  LT  stderr fpr11 000003ffea578832 (f: 3931604992.000000, d: 2.172744e-311)
22:05:50  LT  stderr fpr12 000003ff8c0ff900 (f: 2349857024.000000, d: 2.171963e-311)
22:05:50  LT  stderr fpr13 0000000000000001 (f: 1.000000, d: 4.940656e-324)
22:05:50  LT  stderr fpr14 000003ff8c0bf000 (f: 2349592576.000000, d: 2.171963e-311)
22:05:50  LT  stderr fpr15 0000000000fb0ae0 (f: 16452320.000000, d: 8.128526e-317)
22:05:50  LT  stderr Module=/home/jenkins/workspace/Grinder_iteration_4/openjdkbinary/j2sdk-image/jre/lib/s390x/default/libj9gc_full29.so
22:05:50  LT  stderr Module_base_address=000003FF78D00000
22:05:50  LT  stderr Target=2_90_20230906_725 (Linux 5.14.0-284.25.1.el9_2.s390x)
22:05:50  LT  stderr CPU=s390x (4 logical CPUs) (0x1e5d62000 RAM)
22:05:50  LT  stderr ----------- Stack Backtrace -----------
22:05:50  LT  stderr _ZN23MM_ScavengerRootScanner11doStackSlotEPP8J9ObjectPvPKv+0x10 (0x000003FF78EB4620 [libj9gc_full29.so+0x1b4620])
22:05:50  LT  stderr vmThreadStackDoOSlotIterator+0x30 (0x000003FF78D4A970 [libj9gc_full29.so+0x4a970])
22:05:50  LT  stderr walkBytecodeFrameSlots+0x3ca (0x000003FF87A87172 [libj9vm29.so+0x87172])
22:05:50  LT  stderr walkStackFrames+0x6cc (0x000003FF87A87B8C [libj9vm29.so+0x87b8c])
22:05:50  LT  stderr _ZN28GC_VMThreadStackSlotIterator9scanSlotsEP10J9VMThreadS1_PvPFvP8J9JavaVMPP8J9ObjectS2_P16J9StackWalkStatePKvEbb+0x4e (0x000003FF78D4AAA6 [libj9gc_full29.so+0x4aaa6])
22:05:50  LT  stderr _ZN14MM_RootScanner13scanOneThreadEP18MM_EnvironmentBaseP10J9VMThreadPv+0x14a (0x000003FF78D415DA [libj9gc_full29.so+0x415da])
22:05:50  LT  stderr _ZN14MM_RootScanner11scanThreadsEP18MM_EnvironmentBase+0xe2 (0x000003FF78D3FF5A [libj9gc_full29.so+0x3ff5a])
22:05:50  LT  stderr _ZN14MM_RootScanner9scanRootsEP18MM_EnvironmentBase+0x64 (0x000003FF78D4357C [libj9gc_full29.so+0x4357c])
22:05:50  LT  stderr _ZN12MM_Scavenger24workThreadGarbageCollectEP22MM_EnvironmentStandard+0x34e (0x000003FF78EADFEE [libj9gc_full29.so+0x1adfee])
22:05:50  LT  stderr _ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+0x218 (0x000003FF78E50AB8 [libj9gc_full29.so+0x150ab8])
22:05:50  LT  stderr _Z23dispatcher_thread_proc2P14OMRPortLibraryPv+0x138 (0x000003FF78E4F438 [libj9gc_full29.so+0x14f438])
22:05:50  LT  stderr omrsig_protect+0x366 (0x000003FF879344DE [libj9prt29.so+0x344de])
22:05:50  LT  stderr dispatcher_thread_proc+0x60 (0x000003FF78E4F0B0 [libj9gc_full29.so+0x14f0b0])
22:05:50  LT  stderr thread_wrapper+0x114 (0x000003FF8C085A14 [libj9thr29.so+0x5a14])
22:05:50  LT  stderr start_thread+0x146 (0x000003FF8C39E2F6 [libc.so.6+0x9e2f6])
22:05:50  LT  stderr  (0x000003FF8C32FDBE [libc.so.6+0x2fdbe])
22:05:50  LT  stderr ---------------------------------------
22:05:50  LT  stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2023/09/07 19:05:47 - please wait.

The grinder did have a same failure as

vij-singh commented 10 months ago

@hzongaro FYI

pshipton commented 10 months ago

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_extended.system_s390x_linux_Nightly_testList_1/562 HCRLateAttachWorkload_previewEnabled_1

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_extended.system_s390x_linux_Nightly_testList_1/562/system_test_output.tar.gz

LT  stderr 000003FEB00E3400: Invalid class pointer in stack allocated object in thread Thread-288
LT  stderr 000003FEB00E3400:    O-Slot=000003FEB00ED280
LT  stderr 000003FEB00E3400:    O-Slot value=000003FEB00ED160
LT  stderr 000003FEB00E3400:    PC=000003FFA4619DA0
LT  stderr 000003FEB00E3400:    framesWalked=3
LT  stderr 000003FEB00E3400:    arg0EA=000003FEB00ED288
LT  stderr 000003FEB00E3400:    walkSP=000003FEB00ED260
LT  stderr 000003FEB00E3400:    literals=000003FFA46304D0
LT  stderr 000003FEB00E3400:    jitInfo=0000000000000000
LT  stderr 000003FEB00E3400:    method=000003FFA46304D0 (net/adoptopenjdk/test/hcrAgent/agent/TransformerMakerThread.run()V) (Interpreted)
LT  stderr 000003FEB00E3400:    stack=000003FEB00EBB00-000003FEB00ED2E0
LT  stderr 04:23:49.526 0x3feac002400    j9mm.479    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK17_s390x_linux_Nightly/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
JasonFengJ9 commented 9 months ago

JDK8 x86-32_windows 0.41 milestone 2(win16x64rt2-5)

openjdk version "1.8.0_392"
IBM Semeru Runtime Open Edition (build 1.8.0_392-b07)
Eclipse OpenJ9 VM (build v0.41.0-release-3ede3b5a4, JRE 1.8.0 Windows Server 2016 x86-32-Bit 20231003_756 (JIT enabled, AOT enabled)
OpenJ9   - 3ede3b5a4
OMR      - fa7b6ddc7
JCL      - 6fee1ee642 based on jdk8u392-b07)
[2023-10-03T18:41:20.283Z] variation: Mode1000
[2023-10-03T18:41:20.283Z] JVM_OPTIONS:  

[2023-10-03T18:41:57.244Z] LLT 11:41:33.770 - Completed 6.7%. Number of tests started=5313
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: Object neither in heap nor stack-allocated in thread load-1
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: O-Slot=26D4D6BC
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: O-Slot value=0000000D
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: PC=289091CB
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: framesWalked=1
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: arg0EA=26D4D6C0
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: walkSP=26D4D674
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: literals=00000008
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: jitInfo=275AEB80
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: method=22ED0760 (java/lang/invoke/ConvertHandle.checkConversion(Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;)V) (JIT)
[2023-10-03T18:42:09.084Z] LLT stderr 22F69300: stack=26D47C98-26D4D908
[2023-10-03T18:42:09.084Z] LLT stderr 18:41:45.942 0x22f57100    j9mm.479    *   ** ASSERTION FAILED ** at C:\workspace\openjdk-build\workspace\build\src\openj9\runtime\gc_glue_java\ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
[2023-10-03T18:42:09.084Z] LLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2023/10/03 11:41:45 - please wait.

[2023-10-03T18:42:15.897Z] LangLoadTest_5m_2_FAILED

50x grinder - passed

JasonFengJ9 commented 9 months ago

JDK8 x86-64_mac milestone 2(mac12x64rt15)

openjdk version "1.8.0_392"
IBM Semeru Runtime Open Edition (build 1.8.0_392-b07)
Eclipse OpenJ9 VM (build v0.41.0-release-3ede3b5a4, JRE 1.8.0 Mac OS X amd64-64-Bit Compressed References 20231003_803 (JIT enabled, AOT enabled)
OpenJ9   - 3ede3b5a4
OMR      - fa7b6ddc7
JCL      - 6fee1ee642 based on jdk8u392-b07)

[2023-10-04T20:23:38.600Z] variation: Mode650
[2023-10-04T20:23:38.600Z] JVM_OPTIONS:  -XX:-UseCompressedOops 

[2023-10-04T20:30:28.626Z] TestJlmRemoteClassAuth_1_FAILED

system_test_output.tar.gz

00007F8BDC94C100: Object neither in heap nor stack-allocated in thread load-8
00007F8BDC94C100:       O-Slot=00007F8BDE13C788
00007F8BDC94C100:       O-Slot value=00007F8BE21A11B8
00007F8BDC94C100:       PC=000000001DCFE6DC
00007F8BDC94C100:       framesWalked=1
00007F8BDC94C100:       arg0EA=00007F8BDE13C790
00007F8BDC94C100:       walkSP=00007F8BDE13C6E8
00007F8BDC94C100:       literals=0000000000000010
00007F8BDC94C100:       jitInfo=00007F8BE39D20E8
00007F8BDC94C100:       method=00007F8BDFAF88F8 (java/lang/invoke/ConvertHandle.checkConversion(Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;)V) (JIT)
00007F8BDC94C100:       stack=00007F8BDE135C40-00007F8BDE13CCB0
20:29:27.056 0x7f8bdd893d00    j9mm.479    *   ** ASSERTION FAILED ** at /Users/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-mac-x64-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
JVMDUMP039I Processing dump event "traceassert", detail "" at 2023/10/04 16:29:27 - please wait.
JVMDUMP032I JVM requested System dump using '/Users/jenkins/workspace/Test_openjdk8_j9_sanity.system_x86-64_mac_testList_5/aqa-tests/TKG/output_16964491738654/TestJlmRemoteClassAuth_1/20231004-162337-TestJlmRemoteClassAuth/results/core.20231004.162927.66128.0001.dmp' in response to an event
JVMDUMP010I System dump written to /Users/jenkins/workspace/Test_openjdk8_j9_sanity.system_x86-64_mac_testList_5/aqa-tests/TKG/output_16964491738654/TestJlmRemoteClassAuth_1/20231004-162337-TestJlmRemoteClassAuth/results/core.20231004.162927.66128.0001.dmp
JVMDUMP032I JVM requested Java dump using '/Users/jenkins/workspace/Test_openjdk8_j9_sanity.system_x86-64_mac_testList_5/aqa-tests/TKG/output_16964491738654/TestJlmRemoteClassAuth_1/20231004-162337-TestJlmRemoteClassAuth/results/javacore.20231004.162927.66128.0002.txt' in response to an event
JVMDUMP010I Java dump written to /Users/jenkins/workspace/Test_openjdk8_j9_sanity.system_x86-64_mac_testList_5/aqa-tests/TKG/output_16964491738654/TestJlmRemoteClassAuth_1/20231004-162337-TestJlmRemoteClassAuth/results/javacore.20231004.162927.66128.0002.txt
JVMDUMP032I JVM requested Snap dump using '/Users/jenkins/workspace/Test_openjdk8_j9_sanity.system_x86-64_mac_testList_5/aqa-tests/TKG/output_16964491738654/TestJlmRemoteClassAuth_1/20231004-162337-TestJlmRemoteClassAuth/results/Snap.20231004.162927.66128.0003.trc' in response to an event
JVMDUMP010I Snap dump written to /Users/jenkins/workspace/Test_openjdk8_j9_sanity.system_x86-64_mac_testList_5/aqa-tests/TKG/output_16964491738654/TestJlmRemoteClassAuth_1/20231004-162337-TestJlmRemoteClassAuth/results/Snap.20231004.162927.66128.0003.trc
JVMDUMP013I Processed dump event "traceassert", detail "".

50x grinder - passed

JasonFengJ9 commented 8 months ago

JDK8 s390x_linux(rhel7s390x-svl-rt5-1)

openjdk version "1.8.0_392"
IBM Semeru Runtime Open Edition (build 1.8.0_392-b08)
Eclipse OpenJ9 VM (build v0.41.0-release-a803f004b, JRE 1.8.0 Linux s390x-64-Bit Compressed References 20231116_779 (JIT enabled, AOT enabled)
OpenJ9   - a803f004b
OMR      - 5eee6ad9d
JCL      - b73cbdd342 based on jdk8u392-b08)

[2023-11-16T11:14:07.119Z] variation: Mode650
[2023-11-16T11:14:07.120Z] JVM_OPTIONS:  -XX:-UseCompressedOops 

[2023-11-16T11:15:10.587Z] LT  03:15:07.961 - Completed 6.7%. Number of tests started=994 (+396)
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: Object neither in heap nor stack-allocated in thread Thread-107
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: O-Slot=000003FEB8082270
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: O-Slot value=000003FEA00801E0
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: PC=000003FFB0575E23
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: framesWalked=3
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: arg0EA=000003FEB8082278
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: walkSP=000003FEB8082250
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: Object neither in heap nor stack-allocated in thread Thread-108
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: literals=000003FFB0590168
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: O-Slot=000003FEB8051310
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: jitInfo=0000000000000000
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: O-Slot value=000003FEE000AC70
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: method=000003FFB0590168 (net/adoptopenjdk/test/hcrAgent/agent/TransformerMakerThread.run()V) (Interpreted)
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: PC=000003FFB0575E23
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB807B900: stack=000003FEB8080AF0-000003FEB80822D0
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: framesWalked=3
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: arg0EA=000003FEB8051318
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: walkSP=000003FEB80512F0
[2023-11-16T11:15:20.420Z] LT  stderr 11:15:18.143 0x3fed4002200    j9mm.479    *   000003FEB806D200:   literals=000003FFB0590168
[2023-11-16T11:15:20.420Z] LT  stderr ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))000003FEB806D200:   jitInfo=0000000000000000
[2023-11-16T11:15:20.420Z] LT  stderr 
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: method=000003FFB0590168 (net/adoptopenjdk/test/hcrAgent/agent/TransformerMakerThread.run()V) (Interpreted)
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB806D200: stack=000003FEB804FB90-000003FEB8051370
[2023-11-16T11:15:20.420Z] LT  stderr 11:15:18.143 0x3feb804b200    j9mm.479    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: Object neither in heap nor stack-allocated in thread Thread-109
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: O-Slot=000003FEB807FD80
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: O-Slot value=000003FE7C025400
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: PC=000003FFB0575DEF
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: framesWalked=4
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: arg0EA=000003FEB807FD88
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: walkSP=000003FEB807FD60
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: literals=000003FFB0590168
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: jitInfo=0000000000000000
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: method=000003FFB0590168 (net/adoptopenjdk/test/hcrAgent/agent/TransformerMakerThread.run()V) (Interpreted)
[2023-11-16T11:15:20.420Z] LT  stderr 000003FEB8058000: stack=000003FEB807E600-000003FEB807FDE0
[2023-11-16T11:15:20.420Z] LT  stderr 11:15:18.144 0x3fed0002200    j9mm.479    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9/workspace/build/src/openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
[2023-11-16T11:15:20.420Z] LT  stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2023/11/16 03:15:18 - please wait.

[2023-11-16T11:15:23.878Z] HCRLateAttachWorkload_1_FAILED

50x internal grinder - passed

JasonFengJ9 commented 7 months ago

JDK21 x86-64_windows(win16x86-rtp-rt7-1)

openjdk version "21.0.1" 2023-10-17 LTS
IBM Semeru Runtime Open Edition 21.0.2.0-m2 (build 21.0.1+12-LTS)
Eclipse OpenJ9 VM 21.0.2.0-m2 (build v0.43.0-release-54becab46, JRE 21 Windows Server 2016 amd64-64-Bit Compressed References 20231017_57 (JIT enabled, AOT enabled)
OpenJ9   - 54becab46
OMR      - d810fcb98
JCL      - 10a4ed68a77 based on jdk-21.0.1+12)

[2023-12-15T22:09:53.166Z] variation: Mode610
[2023-12-15T22:09:53.835Z] JVM_OPTIONS:  -Xcompressedrefs -Xjit -Xgcpolicy:gencon 

[2023-12-15T22:11:51.620Z] MT3 14:11:59 >> Loaded 12000 classes...
[2023-12-15T22:11:51.620Z] STF 14:11:59.399 - Found dump at: C:\Users\jenkins\workspace\Test_openjdk21_j9_extended.system_x86-64_windows_testList_2\aqa-tests\TKG\output_17026738541892\SharedClasses.SCM01.MultiThread_1\20231215-141003-SharedClasses\results\core.20231215.141158.7816.0001.dmp
[2023-12-15T22:11:51.620Z] MT4 14:11:59 >> --> Threads 12300 to 12600 <--
[2023-12-15T22:11:51.620Z] MT5 14:11:59 >> --> Threads 12900 to 13200 <--
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: Object neither in heap nor stack-allocated in thread Thread-12558
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: O-Slot=00000072957FF038
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: O-Slot value=0000000001A46B00
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: PC=00007FFD78F1AA70
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: framesWalked=1
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: arg0EA=0000000000909E68
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: walkSP=0000000000909CD8
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: literals=0000000000000000
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: jitInfo=000002341E21FEA8
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: method=00000000000A2AF8 (java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;) (JIT)
[2023-12-15T22:11:51.620Z] MT2 stderr 0000000000416100: stack=00000000009030E8-000000000090A170
[2023-12-15T22:11:51.620Z] MT2 stderr 22:11:58.942 0x416100    j9mm.479    *   ** ASSERTION FAILED ** at c:\workspace\openjdk-build\workspace\build\src\openj9\runtime\gc_glue_java\ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
[2023-12-15T22:11:51.620Z] MT2 stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2023/12/15 14:11:58 - please wait.
[2023-12-15T22:11:51.621Z] MT2 stderr JVMDUMP032I JVM requested System dump using 'C:\Users\jenkins\workspace\Test_openjdk21_j9_extended.system_x86-64_windows_testList_2\aqa-tests\TKG\output_17026738541892\SharedClasses.SCM01.MultiThread_1\20231215-141003-SharedClasses\results\core.20231215.141158.7816.0001.dmp' in response to an event
[2023-12-15T22:11:51.621Z] MT3 14:11:59 >> --> Threads 12000 to 12300 <--
[2023-12-15T22:11:51.621Z] MT3 stderr core file generated - C:\Users\jenkins\workspace\Test_openjdk21_j9_extended.system_x86-64_windows_testList_2\aqa-tests\TKG\output_17026738541892\SharedClasses.SCM01.MultiThread_1\20231215-141003-SharedClasses\results\core.20231215.141158.7816.0001.dmp
[2023-12-15T22:11:51.621Z] MT5 14:11:59 >> Loaded 13000 classes...
[2023-12-15T22:11:52.314Z] MT1 14:11:59 >> --> Threads 12300 to 12600 <--

[2023-12-15T22:12:39.333Z] SharedClasses.SCM01.MultiThread_1_FAILED

50x grinder x86-64_windows SharedClasses.SCM01.MultiThread_1 - 11/50 failed

pshipton commented 7 months ago

grinder with -Xint - passed

@dmitripivkine @hzongaro pls take a look

pshipton commented 7 months ago

The grinder with -Xint passed.

hzongaro commented 6 months ago

Reproduced with internal grinder run 37044, using EXTRA_OPTIONS

-XX:+MergeCompilerOptions -Xjit:{java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic*}(traceFull,log=resolveinvokedynamic.log)

From that run:

18:15:38  MT3 stderr 0000000000521E00: Object neither in heap nor stack-allocated in thread Thread-16815
18:15:38  MT3 stderr 0000000000521E00:  O-Slot=00000004252FEA98
18:15:38  MT3 stderr 0000000000521E00:  O-Slot value=000000000220CB00
18:15:38  MT3 stderr 0000000000521E00:  PC=00007FF8E6FECCEC
18:15:38  MT3 stderr 0000000000521E00:  framesWalked=1
18:15:38  MT3 stderr 0000000000521E00:  arg0EA=0000000000ED7A18
18:15:38  MT3 stderr 0000000000521E00:  walkSP=0000000000ED7858
18:15:38  MT3 stderr 0000000000521E00:  literals=0000000000000000
18:15:38  MT3 stderr 0000000000521E00:  jitInfo=0000019BB803A428
18:15:38  MT3 stderr 0000000000521E00:  method=00000000000A2AF8 (java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;) (JIT)
18:15:38  MT3 stderr 0000000000521E00:  stack=0000000000ED0C58-0000000000ED7D10
18:15:38  MT3 stderr 23:15:37.599 0x1c4300    j9mm.479    *   ** ASSERTION FAILED ** at c:\workspace\openjdk-build\workspace\build\src\openj9\runtime\gc_glue_java\ScavengerRootScanner.hpp:109: ((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))

From the JIT trace log ./aqa-tests/TKG/output_170294115796/SharedClasses.SCM01.MultiThread_1/20231218-151254-SharedClasses/results/resolveinvokedynamic.log.2.6104.41220.20231218.151340.6104, that point in the code corresponds to a call to jitNewArray following outlined label L0338. There is a branch to that outlined label from block_39:

0x00007FF8E6FE4C46 0000025e [0x00007FF4D95F8420]                            fence Relative [ 0x00007FF4DEE63770 ]       ; fence BBStart <block_39> (frequency 194)

 \\ java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;
 \\   16 JBinvokestatic 20 java/lang/invoke/MethodHandleResolver.getClassFromJ9Class(J)Ljava/lang/Class;
 \\      26 JBlload0

0x00007FF8E6FE4C46 0000025e [0x00007FF4D95F8840] 4c 8b 8c 24 90 01 00 00    mov r9, qword ptr [rsp+0x190]               ; L8RegMem, SymRef  j9class<parm 0 J>[#4101  Parm +400] [flags 0xc0000104 0x0 ]

 \\ java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;
 \\   16 JBinvokestatic 20 java/lang/invoke/MethodHandleResolver.getClassFromJ9Class(J)Ljava/lang/Class;
 \\      27 JBinvokevirtual 82 com/ibm/jit/JITHelpers.getClassFromJ9Class64(J)Ljava/lang/Class;

0x00007FF8E6FE4C4E 00000266 [0x00007FF4D95F8980] 49 8b 79 30                mov rdi, qword ptr [r9+0x30]                ; L8RegMem, SymRef  <javaLangClassFromClass>[#336  Shadow +48] [flags 0x607 0x0 ]

 \\ java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;
 \\   19 JBastore 8

0x00007FF8E6FE4C52 0000026a [0x00007FF4D95F8DB0] 48 89 bc 24 90 00 00 00    mov qword ptr [rsp+0x90], rdi               ; S8MemReg, SymRef  classObject<auto slot 8>[#4102  Auto +144] [flags 0x4007 0x0 ]

 \\ java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;
 \\   16 JBinvokestatic 20 java/lang/invoke/MethodHandleResolver.getClassFromJ9Class(J)Ljava/lang/Class;
 \\      30 JBastore2

0x00007FF8E6FE4C5A 00000272 [0x00007FF4D95F9170] 48 89 bc 24 e8 00 00 00    mov qword ptr [rsp+0xe8], rdi               ; S8MemReg, SymRef  classObject<auto slot 2>[#4103  Auto +232] [flags 0x4007 0x0 ]
0x00007FF8E6FE4C62 0000027a [0x00007FF4D95F9750]                            Label L0336:                        ; label ; (Start of internal control flow)

 \\ java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;
 \\   25 JBanewarray 15 java/lang/Object

0x00007FF8E6FE4C62 0000027a [0x00007FF4D95FBB40] 48 8b 45 60                mov rax, qword ptr [rbp+0x60]               ; L8RegMem, SymRef [#4106 +96]
0x00007FF8E6FE4C66 0000027e [0x00007FF4D95FBC60] 4c 8d 60 10                lea r12, qword ptr [rax+0x10]               ; LEA8RegMem, SymRef [#4107 +16]
0x00007FF8E6FE4C6A 00000282 [0x00007FF4D95FBD80] 4c 3b 65 68                cmp r12, qword ptr [rbp+0x68]               ; CMP8RegMem, SymRef [#4108 +104]
0x00007FF8E6FE4C6E 00000286 [0x00007FF4D8086410]                            vfpSave                     ; AdjustFramePtr
0x00007FF8E6FE4C6E 00000286 [0x00007FF4D95FBE10] 0f 87 53 80 00 00          ja  Outlined Label L0338                    ; JA4

...

0x00007FF8E6FECCC7 000082df [0x00007FF4D95F98E0]                            Outlined Label L0338:                       ; label

 \\ java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;
 \\    0 JBinvokestatic 16 com/ibm/oti/vm/VM.getVMLangAccess()Lcom/ibm/oti/vm/VMLangAccess;

0x00007FF8E6FECCC7 000082df [0x00007FF4D80877A0] 49 8b d9                   mov rbx, r9         ; MOV8RegReg
0x00007FF8E6FECCCA 000082e2 [0x00007FF4D8087710] 48 8b f0                   mov rsi, rax                ; MOV8RegReg

 \\ java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;
 \\   24 JBiconst2

0x00007FF8E6FECCCD 000082e5 [0x00007FF4D95F9A40] ba 02 00 00 00             mov edx, 0x00000002 ; MOV4RegImm4

 \\ java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;
 \\   25 JBanewarray 15 java/lang/Object

0x00007FF8E6FECCD2 000082ea [0x00007FF4D8378A30] 48 b8 00 e9 02 00 00 00 00 mov rax, 0x000000000002e900 ; MOV8RegImm64
0x00007FF8E6FECCDC 000082f4 [0x00007FF4D95F9C60] 44 8d 04 05 00 00 00 00    lea r8d, dword ptr [1*rax]          ; LEA4RegMem
0x00007FF8E6FECCE4 000082fc [0x00007FF4D95FA9D0] 48 8b cd                   mov rcx, rbp                ; MOV8RegReg
0x00007FF8E6FECCE7 000082ff [0x00007FF4D95FB140]                            assocreg                        ; assocreg
0x00007FF8E6FECCE7 000082ff [0x00007FF4D95FAF00] e8 14 8e c9 13             call        jitANewArray            ; CALLImm4 (0x00007FF8FAC85B00); CALLImm4

The stack atlas for those two sections of code shows a collected reference in ebx at the call to jitNewArray. The contents of that register were copied from r9 at 0x00007FF8E6FECCC7. In block_39, the value in r9 was loaded at 0x00007FF8E6FE4C46 from j9class<parm 0 J>[#4101 Parm +400]. r9 is marked as containing a collected reference in that block, as can be seen below. I assume the j9class symbol should not have been marked as containing a collected reference. Now trying to understand how that came about.

Stack atlas for block_39:

  Map number : 429
  Code offset range starts at [00000266]
  GC stack map information :
    number of stack slots mapped = 27
    live stack slots containing addresses --> {0,1,2,3,6,18}
  GC register map information :
    slot pushes: 0    registers: {r9d }

Stack atlas for outlined code at L0038:

  Map number : 30
  Code offset range starts at [000082ff]
  GC stack map information :
    number of stack slots mapped = 27
    live stack slots containing addresses --> {0,1,2,3,4,6,15,18}
  GC register map information :
    slot pushes: 0    registers: {ebx edi }
hzongaro commented 6 months ago

Looking at Instruction Selection, we see this for code in block_39. Notice j9class is loaded in GPR_0160, which is not marked as a collected reference.

; Live regs: GPR=0 FPR=0 VRF=0 {}
------------------------------
 n24065n  (  0)  NULLCHK on n24063n [#32] ()                                                          [0x00007FF4DED351E0] bci=[5,27,288] rc=0 vc=13 vn=- li=39 udi=- nc=1 flg=0x8
 n24064n  (  4)    aloadi  <javaLangClassFromClass>[#336  Shadow +48] [flags 0x607 0x0 ] (X!=0 X>=0 )  [0x00007FF4DED35190] bci=[5,27,288] rc=4 vc=13 vn=- li=39 udi=- nc=1 flg=0x10c
 n24063n  (  2)      lload  j9class<parm 0 J>[#421  Parm] [flags 0xc0000104 0x0 ] (X>=0 cannotOverflow )  [0x00007FF4DED35140] bci=[5,26,288] rc=2 vc=13 vn=- li=39 udi=- nc=0 flg=0x1100
------------------------------
------------------------------
 n24065n  (  0)  NULLCHK on n24063n [#32] ()                                                          [0x00007FF4DED351E0] bci=[5,27,288] rc=0 vc=13 vn=- li=39 udi=- nc=1 flg=0x8
 n24064n  (  3)    aloadi  <javaLangClassFromClass>[#336  Shadow +48] [flags 0x607 0x0 ] (in &GPR_0161) (X!=0 X>=0 )  [0x00007FF4DED35190] bci=[5,27,288] rc=3 vc=13 vn=- li=39 udi=35072 nc=1 flg=0x10c
 n24063n  (  1)      lload  j9class<parm 0 J>[#421  Parm] [flags 0xc0000104 0x0 ] (in GPR_0160) (X!=0 X>=0 cannotOverflow )  [0x00007FF4DED35140] bci=[5,26,288] rc=1 vc=13 vn=- li=39 udi=34752 nc=0 flg=0x1104
------------------------------

A little later in instruction selection, GPR_0160 becomes a collected reference during evaluation of the following ificmpeq tree. I suspect that the problem lies in l2aEvaluator, which thinks that the l2a is being used for a compressed reference with a shift amount of zero, and that the operand must be a collected reference. Similar l2aEvaluator methods exist for other hardware platforms.

============================================================
; Live regs: GPR=2 FPR=0 VRF=0 {&GPR_0253, GPR_0160}
------------------------------
 n30144n  (  0)  ificmpeq --> block_199 BBStart at n24578n ()                                         [0x00007FF4DE01BE00] bci=[10,9,1718] rc=0 vc=13 vn=- li=524 udi=- nc=2 flg=0x20
 n30139n  (  1)    iand (X>=0 cannotOverflow )                                                        [0x00007FF4DE01BC70] bci=[10,6,1718] rc=1 vc=13 vn=- li=524 udi=- nc=2 flg=0x1100
 n30140n  (  1)      iloadi  <isArray>[#342  Shadow +16] [flags 0x603 0x0 ] (cannotOverflow )         [0x00007FF4DE01BCC0] bci=[10,6,1718] rc=1 vc=13 vn=- li=524 udi=- nc=1 flg=0x1000
 n30141n  (  1)        aloadi  <classRomPtr>[#335  Shadow +8] [flags 0x10607 0x0 ] (X>=0 )            [0x00007FF4DE01BD10] bci=[10,6,1718] rc=1 vc=13 vn=- li=524 udi=- nc=1 flg=0x100
 n36479n  (  2)          l2a (X>=0 )                                                                  [0x00007FF4DAFC7A30] bci=[5,26,288] rc=2 vc=13 vn=- li=524 udi=- nc=1 flg=0x100
 n24063n  (  1)            ==>lload (in GPR_0160) (X!=0 X>=0 cannotOverflow )
 n30142n  (  2)      iconst 0x20000 (X!=0 X>=0 )                                                      [0x00007FF4DE01BD60] bci=[10,6,1718] rc=2 vc=13 vn=- li=524 udi=- nc=0 flg=0x104
 n30142n  (  2)    ==>iconst 0x20000 (X!=0 X>=0 )
------------------------------
[ 11655] O^O LogicalEvaluator: checking that the store has not happened yet. Target register: 0
[ 11656] O^O checking that the store has not happened yet. Target register: 0
------------------------------
 n30144n  (  0)  ificmpeq --> block_199 BBStart at n24578n ()                                         [0x00007FF4DE01BE00] bci=[10,9,1718] rc=0 vc=13 vn=- li=524 udi=- nc=2 flg=0x20
 n30139n  (  0)    iand (in GPR_0257) (X>=0 cannotOverflow )                                          [0x00007FF4DE01BC70] bci=[10,6,1718] rc=0 vc=13 vn=- li=524 udi=11664 nc=2 flg=0x1100
 n30140n  (  0)      iloadi  <isArray>[#342  Shadow +16] [flags 0x603 0x0 ] (in GPR_0257) (cannotOverflow )  [0x00007FF4DE01BCC0] bci=[10,6,1718] rc=0 vc=13 vn=- li=524 udi=11664 nc=1 flg=0x1000
 n30141n  (  0)        aloadi  <classRomPtr>[#335  Shadow +8] [flags 0x10607 0x0 ] (in GPR_0256) (X>=0 )  [0x00007FF4DE01BD10] bci=[10,6,1718] rc=0 vc=13 vn=- li=524 udi=11392 nc=1 flg=0x100
 n36479n  (  1)          l2a (in &GPR_0160) (X>=0 )                                                   [0x00007FF4DAFC7A30] bci=[5,26,288] rc=1 vc=13 vn=- li=524 udi=34752 nc=1 flg=0x100
 n24063n  (  0)            ==>lload (in &GPR_0160) (X!=0 X>=0 cannotOverflow )
 n30142n  (  0)      iconst 0x20000 (X!=0 X>=0 )                                                      [0x00007FF4DE01BD60] bci=[10,6,1718] rc=0 vc=13 vn=- li=524 udi=- nc=0 flg=0x104
 n30142n  (  0)    ==>iconst 0x20000 (X!=0 X>=0 )
------------------------------
pshipton commented 6 months ago

Ran a grinder on jdk17 head stream and the failure occurs there as well, it's not specific to jdk21. I ran jdk21 xlinux and the failure didn't occur (or takes more than 75 iterations). Ran a grinder on jdk17 0.41 and the failure doesn't occur, it's a regression from the last release. Ran a grinder on jdk17 0.43 M1 and the failure occurs.

aprenaud commented 6 months ago

@hzongaro any update Henry?

hzongaro commented 6 months ago

Although the problem could not be reproduced with JDK 17 0.41, I don't believe this is a regression. I believe this problem has existed for a very long time in the JIT compiler, and could occur with compressed references enabled with a shift amount of zero. I suspect that a recent change to MethodHandleResolver.resolveInvokeDynamic has made it more likely that the JIT would encounter the problem.

I am still working on a fix.

vijaysun-omr commented 6 months ago

Discussing with Henry, I feel we should probably prevent commoning of l2a as the safe fix at this stage of the release. Later, Henry will probably look at whether a better fix is possible in l2aEvaluator for example, such that we don't use a collected reference register for l2a (we may need to check other uses of l2a and also think about the semantics of the opcode before we do such a change though).

hzongaro commented 6 months ago

Following up on Vijay's comment, I came across past discussion of the semantics of l2a from the OMR Architecture Meeting of May 26, 2022. Details of the discussion and decision can be found in eclipse/omr#6508.

I'll quote in passing a relevant comment from @jdmpapin from that OMR issue:

However, the issue isn't really commoning. It's whether there is a GC point between the instruction that sets the register and any later instruction that consumes the register. In the first set of pasted trees, typeOfBuffer is unresolved, so evaluation of n4145n has generated a sequence that first gets the base object reference into a register (GPR_0755) and then resolves the field (GC point) before using it.

The same is true here. In the interim I will try to work around the problem by preventing commoning of the l2a, as Vijay has suggested, and hopefully that will avoid the issue until the new l2gcref proposed in eclipse/omr#6508 is implemented.

hzongaro commented 6 months ago

I have opened OMR pull request eclipse/omr#7217, which should work around the problem that is causing SharedClasses.SCM01.MultiThread_1 to fail. Other failures will need further investigation.

pshipton commented 6 months ago

@hzongaro pls create PRs for 0.42 and 043.

pshipton commented 6 months ago

No concerns in the jdk21 nightly build https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-JDK21/197/

Internal grinder on SharedClasses.SCM01.MultiThread_1 failed 2/50.

OpenJ9 grinder https://openj9-jenkins.osuosl.org/view/Test/job/Grinder/3161 passed 50/50.

@hzongaro I'll leave it up to you if there is something else that can be resolved in 0.43 or if this should be moved forward.

hzongaro commented 6 months ago

Internal grinder on SharedClasses.SCM01.MultiThread_1 failed 2/50.

The failure symptom appears to be different, so I suspect something else is going on there. I'll move investigation of that and the failures reported earlier in HCRLateAttachWorkload, TestJlmRemoteClassAuth and LangLoadTest_5m to 0.44.

hzongaro commented 6 months ago

The failure symptom appears to be different

I somehow confused myself - the failures in the internal grinder still look like the same problem:

10:11:40  STF 07:11:48.363 - Found dump at: C:\Users\jenkins\workspace\Grinder_iteration_4\aqa-tests\TKG\output_17047266223945\SharedClasses.SCM01.MultiThread_1\20240108-071042-SharedClasses\results\core.20240108.071148.8520.0001.dmp
10:11:40  MT3 stderr 00000000004EA100: Object neither in heap nor stack-allocated in thread Thread-6912
10:11:40  MT3 stderr 00000000004EA100:  O-Slot=000000DCB32FEED8
10:11:40  MT3 stderr 00000000004EA100:  O-Slot value=00000000011C9200
10:11:40  MT3 stderr 00000000004EA100:  PC=00007FF6A75F0364
10:11:40  MT3 stderr 00000000004EA100:  framesWalked=1
10:11:40  MT3 stderr 00000000004EA100:  arg0EA=00000000009000C8
10:11:40  MT3 stderr 00000000004EA100:  walkSP=00000000008FFF58
10:11:40  MT3 stderr 00000000004EA100:  literals=0000000000000000
10:11:40  MT3 stderr 00000000004EA100:  jitInfo=000002D0F243DA28
10:11:40  MT3 stderr 00000000004EA100:  method=00000000000A1E08 (java/lang/invoke/MethodHandleResolver.resolveInvokeDynamic(JLjava/lang/String;Ljava/lang/String;J)Ljava/lang/Object;) (JIT)
10:11:40  MT3 stderr 00000000004EA100:  stack=00000000008F9340-00000000009003A0

I'll move this back to 0.43 and investigate further now.

hzongaro commented 6 months ago

The grinder failures reported after applying pull request eclipse/omr#7217 involve a variant that wasn't caught by the earlier workaround. From a subsequent grinder run with tracing enabled for MethodHandleResolver.resolveInvokeDynamic, the problematic register can ultimately be traced back to this point during tree evaluation, where the child of an l2a — the lRegLoad — has its register marked as a collected reference:

 n39287n  (  0)  BBStart <block_57> (freq 199)                  [0x00007FF4129DE7E0] bci=[-1,16,167] rc=0 vc=13 vn=- li=57 udi=- nc=1
 n54664n  (  1)    GlRegDeps                                    [0x00007FF40F01AE60] bci=[-1,16,167] rc=1 vc=13 vn=- li=57 udi=- nc=1
 n54665n  (  5)      lRegLoad eax   j9class<parm 0 J>[#421  Parm] [flags 0xc0000104 0x0 ] (X>=0 cannotOverflow SeenRealReference )  [0x00007FF40F01AEB0] bci=[5,26,315] rc=5 vc=13 vn=- li=57 udi=- nc=0 flg=0x9100
...
============================================================
; Live regs: GPR=2 FPR=0 VRF=0 {&GPR_0205, GPR_0128}
------------------------------
 n51498n ( 0) treetop                                       [0x00007FF41048D0C0] bci=[10,6,1718] rc=0 vc=13 vn=- li=764 udi=- nc=1
 n54215n ( 1)  l2a                                        [0x00007FF40F012210] bci=[5,26,315] rc=1 vc=13 vn=- li=764 udi=- nc=1
 n54665n ( 3)   ==>lRegLoad (in GPR_0128) (X!=0 X>=0 cannotOverflow SeenRealReference )
------------------------------
------------------------------
 n51498n ( 0) treetop                                       [0x00007FF41048D0C0] bci=[10,6,1718] rc=0 vc=13 vn=- li=764 udi=- nc=1
 n54215n ( 0)  l2a (in &GPR_0128)                                 [0x00007FF40F012210] bci=[5,26,315] rc=0 vc=13 vn=- li=764 udi=52336 nc=1
 n54665n ( 2)   ==>lRegLoad (in &GPR_0128) (X!=0 X>=0 cannotOverflow SeenRealReference )
------------------------------

The earlier pull request worked around the problem if the l2a appeared in the context of a memory reference operation, but that is not the case here, so that workaround didn't avoid this instance of the problem. The l2a in both cases is introduced by Tree Simplification in the process of simplifying a pair of indirect references that ultimately yield the Int64 value used in the first indirect reference. That value is converted to an address using an l2a operation.

Following discussion with @vijaysun-omr and @0xdaryl, I attempted test runs (sanity.functional, extended.functional, sanity.system, extended.system and sanity.openjdk) to see how frequently this transformation is applied. Those runs didn't encounter any instances in which the transformation was performed - though we do know that it happens intermittently for SharedClasses.SCM01.MultiThread_1.

Given that the transformation appears to be applied very infrequently, we agreed that the best thing to do for now would be to simply disable this transformation in the main branch and in the 0.43 release to prevent this problematic IL arising. The earlier OMR pull requests already greatly reduce the likelihood of this problem arising, so the initial release build JDK 21 (release 0.42) should be safe without preventing the transformation.

I will follow up with those pull requests.

hzongaro commented 6 months ago

Opened pull request #18743 to disable transformation in Tree Simplification in the main branch.

hzongaro commented 6 months ago

Opened pull request #18747 to disable transformation in Tree Simplification in the 0.43.0-release branch.

hzongaro commented 6 months ago

Moved further investigation out to Release 0.44

hzongaro commented 5 months ago

I would like to delay making the changes to distinguish between collected and non-collected addresses in integer to address conversion operations so that they can go through a longer test cycle. Moving this out to the 0.45 release.