search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.27k stars 721 forks source link

[JDK16/Aarch64] Assertion failure occurs frequently in GC during compilation #12408

Open ChengJin01 opened 3 years ago

ChengJin01 commented 3 years ago

Steps to reproduce the crash by following the instruction at https://github.com/eclipse/openj9/blob/master/doc/build-instructions/Build_Instructions_V16.md#aarch64

export JAVA_HOME=../aarch64_jdk (downloaded from AdoptOpenJDK) export PATH="$JAVA_HOME/bin:$PATH"

git clone https://github.com/ibmruntimes/openj9-openjdk-jdk16.git cd openj9-openjdk-jdk16 git revert 5d554a114d64937d59f432c5e0ac3b4e6e691531 (OpenJ9 support for the Foreign Memory Access API) bash get_source.sh bash configure --disable-warnings-as-errors --enable-openjdk-methodhandles make all

It ended up with the following dump:

...
[100%] Built target j9ddr

ERROR: Build failed for target 'all' in configuration 'linux-aarch64-server-release' (exit code 2)
=== Output from failing command(s) repeated here ===
* For target support_gensrc_jdk.localedata__cldr-gensrc.marker:
0000000000019200: Invalid class pointer in stack allocated object in thread main
0000000000019200:       O-Slot=000000000010F408
0000000000019200:       O-Slot value=000000000010F220
0000000000019200:       PC=00004000290F009C
0000000000019200:       framesWalked=7
0000000000019200:       arg0EA=000000000010F460
0000000000019200:       walkSP=000000000010CAE8
0000000000019200:       literals=0000000000000000
0000000000019200:       jitInfo=000040003B7D0038
0000000000019200:       method=00000000001DBDF8 (build/tools/cldrconverter/ResourceBundleGenerator.generateBundle(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;ZLjava/util/Map;Lbuild/tools/cldrconverter/BundleGenerator$BundleType;)V) (JIT)
0000000000019200:       stack=0000000000109700-000000000010F5F0
16:27:22.758 0x19200    j9mm.479    *   ** ASSERTION FAILED ** at ../../../../../../openj9/runtime/gc_glue_java/ScavengerRootScanner.hpp:108: 
((MM_StackSlotValidator(MM_StackSlotValidator::NOT_ON_HEAP, *slotPtr, stackLocation, walkState).validate(_env)))
JVMDUMP039I Processing dump event "traceassert", detail "" at 2021/04/10 12:27:22 - please wait.
JVMDUMP032I JVM requested System dump using '/.../openj9-openjdk-jdk16/make
/core.20210410.122722.23666.0001.dmp' in response to an event
JVMDUMP010I System dump written to /.../openj9-openjdk-jdk16/make/core.20210410.122722.23666.0001.dmp
   ... (rest of output omitted)

along with the backtrace:

(gdb) bt
#0  0x00004000182ed02c in pthread_kill () from /lib64/libpthread.so.0
#1  0x0000400018bd8654 in omrdump_create ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9prt29.so
#2  0x0000400018e68910 in doSystemDump ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9dmp29.so
#3  0x0000400018e64770 in protectedDumpFunction ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9dmp29.so
#4  0x0000400018bda238 in omrsig_protect ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9prt29.so
#5  0x0000400018e67d68 in runDumpFunction ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9dmp29.so
#6  0x0000400018e67f10 in runDumpAgent ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9dmp29.so
#7  0x0000400018e7f1b4 in triggerDumpAgents ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9dmp29.so
#8  0x0000400018edf188 in triggerHit ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9trc29.so
#9  0x0000400018ed228c in doTracePoint ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9trc29.so
#10 0x0000400018eca138 in javaTrace ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9trc29.so
#11 0x0000400019a83928 in MM_ScavengerRootScanner::doStackSlot(J9Object**, void*, void const*) ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9gc29.so
#12 0x00004000192871f0 in walkJITFrameSlotsForInternalPointers ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9jit29.so
#13 0x0000400019686e80 in walkJITFrameSlots.isra.5 ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9jit29.so
#14 0x00004000196872dc in jitWalkFrame ()
---Type <return> to continue, or q <return> to quit---
   from /.../jdk16_openj9_aarch64/lib/default/libj9jit29.so
#15 0x00004000196880c0 in jitWalkStackFrames ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9jit29.so
#16 0x0000400018a53aa4 in walkStackFrames ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9vm29.so
#17 0x00004000199422b8 in GC_VMThreadStackSlotIterator::scanSlots(J9VMThread*, J9VMThread*, void*, void (*)(J9JavaVM*, J9Object**, void*, J9StackWalkState*, void const*), bool, bool) ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9gc29.so
#18 0x00004000199422b8 in GC_VMThreadStackSlotIterator::scanSlots(J9VMThread*, J9VMThread*, void*, void (*)(J9JavaVM*, J9Object**, void*, J9StackWalkState*, void const*), bool, bool) ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9gc29.so
#19 0x00004000199422b8 in GC_VMThreadStackSlotIterator::scanSlots(J9VMThread*, J9VMThread*, void*, void (*)(J9JavaVM*, J9Object**, void*, J9StackWalkState*, void const*), bool, bool) ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9gc29.so
#20 0x00004000199422b8 in GC_VMThreadStackSlotIterator::scanSlots(J9VMThread*, J9VMThread*, void*, void (*)(J9JavaVM*, J9Object**, void*, J9StackWalkState*, void const*), bool, bool) ()
   from /.../jdk16_openj9_aarch64/lib/default/libj9gc29.so
...

FYI: @dmitripivkine

dmitripivkine commented 3 years ago

Bad O-slot ("Invalid class pointer in stack allocated object") in JIT frame @0xdaryl FYI

0xdaryl commented 3 years ago

Thanks for the investigation. @knn-k @Akira1Saitoh , would one of you mind investigating please?

knn-k commented 3 years ago

Which version did you use as the boot JDK here?

12279 and #12314 may have already fixed it. Try using the ver.0.26.0 M2 binary as the boot JDK.

ChengJin01 commented 3 years ago

Which version did you use as the boot JDK here?

12279 and #12314 may have already fixed it. Try using the ver.0.26.0 M2 binary as the boot JDK.

The boot JDK is downloaded from the nightly builds at https://adoptopenjdk.net/nightly.html?variant=openjdk16&jvmVariant=openj9

Akira1Saitoh commented 3 years ago

I tried to reproduce it on ub18-aarch64-8, but not successful so far. I used the following boot JDK:

OpenJDK Runtime Environment AdoptOpenJDK-16+36-202104092334 (build 16+36-202104092334) Eclipse OpenJ9 VM AdoptOpenJDK-16+36-202104092334 (build master-9a9df50b9, JRE 16 Linux aarch64-64-Bit Compressed References 20210409_16 (JIT enabled, AOT enabled) OpenJ9 - 9a9df50b9 OMR - e0b3f5c8b JCL - 073accb86d based on jdk-16+36)

ChengJin01 commented 3 years ago

I tried to reproduce it on ub18-aarch64-8, but not successful so far. I used the following boot JDK:

OpenJDK Runtime Environment AdoptOpenJDK-16+36-202104092334 (build 16+36-202104092334) Eclipse OpenJ9 VM AdoptOpenJDK-16+36-202104092334 (build master-9a9df50b9, JRE 16 Linux aarch64-64-Bit Compressed References 20210409_16 (JIT enabled, AOT enabled) OpenJ9 - 9a9df50 OMR - e0b3f5c8b JCL - 073accb86d based on jdk-16+36)

It failed on cent7-aarch64 (e.g. cent7-aarch64-6)

Akira1Saitoh commented 3 years ago

I could not reproduce it on cent7-aarch64-4 either. The boot JDK was the same one as I used on ub18-aarch64-8, and I got it from: https://github.com/AdoptOpenJDK/openjdk16-binaries/releases/download/jdk16u-2021-04-12-18-50/OpenJDK16U-jdk_aarch64_linux_openj9_2021-04-12-18-50.tar.gz

The SHA hash of the source tree:

OpenJ9 - 9b90f0a09 OMR - 440f0991d JCL - f06632879 based on jdk-16+36 (git revert 5d554a114d64937d59f432c5e0ac3b4e6e691531 on 073accb)

ChengJin01 commented 3 years ago

It seems weird that I was unable to reproduce the assertion failure even on the same source (previously failed) without any updated code on cent7-aarch64-4 (tried to compile a couple of times but nothing wrong happened)

...
Updating support/demos/image/jfc/TransparentRuler/src.zip
openjdk version "16-internal" 2021-03-16
OpenJDK Runtime Environment (build 16-internal+0-adhoc.jenkins.openj9-openjdk-jdk16)
Eclipse OpenJ9 VM (build master-ee83667b9, JRE 16 Linux aarch64-64-Bit Compressed References 20210413_000000 (JIT enabled, AOT enabled)
OpenJ9   - ee83667b9
OMR      - d7a31696c
JCL      - efa722a9637 based on jdk-16+36)
Creating java.base.jmod
Creating support/demos/image/jfc/CodePointIM/CodePointIM.jar
Creating support/demos/image/jfc/FileChooserDemo/FileChooserDemo.jar
Creating support/demos/image/jfc/Font2DTest/Font2DTest.jar
Creating support/demos/image/jfc/Metalworks/Metalworks.jar
Creating support/demos/image/jfc/Notepad/Notepad.jar
Creating support/demos/image/jfc/Stylepad/Stylepad.jar
Creating support/demos/image/jfc/SampleTree/SampleTree.jar
Creating support/demos/image/jfc/TableExample/TableExample.jar
Creating support/demos/image/jfc/TransparentRuler/TransparentRuler.jar
Creating support/demos/image/jfc/SwingSet2/SwingSet2.jar
Creating support/demos/image/jfc/J2Ddemo/J2Ddemo.jar
Creating jdk image
Stopping sjavac server
Finished building target 'all' in configuration 'linux-aarch64-server-release'
[jenkins@cent7-aarch64-6 openj9-openjdk-jdk16]$

We'd better leave this issue open for a while in case it reoccurs somewhere else.