search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.25k stars 715 forks source link

VM build failure - VMState: 0x00050080 or 0x000501ff #9416

Closed M-Davies closed 3 years ago

M-Davies commented 4 years ago

Got this on https://ci.adoptopenjdk.net/view/Failing%20Builds/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-mac-x64-openj9/573 (build-macstadium-macos1014-1)

* For target support_jmods_java.compiler.jmod:
Assertion failed at /Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/build/macosx-x86_64-normal-server-release/vm/compiler/../compiler/compile/J9SymbolReferenceTable.cpp:811: containingClass != NULL
VMState: 0x00050080
    failed to get defining class of field ref cpIndex=35 in owning method J9Method=0xc5e3660
compiling jdk/internal/org/objectweb/asm/ClassWriter.<init>(I)V at level: warm

JVMDUMP039I Processing dump event "abort", detail "" at 2020/04/30 00:59:05 - please wait.
JVMDUMP032I JVM requested System dump using '/Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/make/core.20200430.005905.52000.0001.dmp' in response to an event
JVMDUMP012E Error in System dump: The core file created by child process with pid = 52079 was not found. Expected to find core file with name "/cores/core.52079"
JVMDUMP032I JVM requested Java dump using '/Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/make/javacore.20200430.005905.52000.0002.txt' in response to an event
JVMDUMP010I Java dump written to /Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/make/javacore.20200430.005905.52000.0002.txt
JVMDUMP032I JVM requested Snap dump using '/Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/make/Snap.20200430.005905.52000.0003.trc' in response to an event
JVMDUMP010I Snap dump written to /Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/make/Snap.20200430.005905.52000.0003.trc
JVMDUMP007I JVM Requesting JIT dump using '/Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/make/jitdump.20200430.005905.52000.0004.dmp'
JVMDUMP010I JIT dump written to /Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/make/jitdump.20200430.005905.52000.0004.dmp
JVMDUMP013I Processed dump event "abort", detail "".

* All command lines available in /Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/build/macosx-x86_64-normal-server-release/make-support/failure-logs.

Access to the machine can be granted by opening an issue at https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues

pshipton commented 4 years ago

Also seen here https://github.com/eclipse/openj9/issues/9291

DanHeidinga commented 4 years ago

@andrewcraik @gita-omr Can someone take a look at this? It seems to be a newish assert failure. This one occurred on osx, #9291 on aix ~10 days ago.

andrewcraik commented 4 years ago

@cathyzhyi might this be related to the AOT issues we have been looking at?

cathyzhyi commented 4 years ago

This failure seems very likely a owning method and cpIndex mismatch issue but we would need a backtrace to tell the root cause.

cathyzhyi commented 4 years ago

assigned to @Leonardo2718

DanHeidinga commented 4 years ago

@Leonardo2718 any update on this? We're coming up on the v0.21.0 code split on June 7

Leonardo2718 commented 4 years ago

So, I wasn't able to get very far because I couldn't get the build that the assert was triggered in. There's really not a lot that can be done with the available information, so the best thing to do is to try to capture more information the next time this failure is seen. I've been trying to think of a way of improving the assert message but haven't found one yet.

DanHeidinga commented 4 years ago

https://github.com/eclipse/openj9/issues/9291 seems very similar. Is that easier to reproduce to debug?

Leonardo2718 commented 4 years ago

No, #9291 has the same problem, unfortunately.

DanHeidinga commented 4 years ago

The same problem = can't get the build?

Are you able to run the test locally or on a cloud machine to reproduce the issue? We have the nightly builds and debug images that can be downloaded from Adopt, especially for recent builds so reproducing this - locally or on a build farm - seems like the next step to me.

Leonardo2718 commented 4 years ago

So, the problem is that the failure is happening during the build, which makes reproducing the issue and extracting useful information very time-consuming. I'm going to try requesting access to the machine at Adopt to see if I have better luck reproducing the failure. It's probably still going to take a lot of time to figure this out, though.

DanHeidinga commented 4 years ago

Any update on this @Leonardo2718? Do you expect it to make the 0.22.0 code split on Aug 9?

Leonardo2718 commented 4 years ago

@DanHeidinga

Any update on this @Leonardo2718? Do you expect it to make the 0.22.0 code split on Aug 9?

Unfortunately, I haven't been able to reproduce this so there's no chance of it making the 0.22 code split.

DanHeidinga commented 4 years ago

I've moved it to the 0.23 milestone

Leonardo2718 commented 3 years ago

From https://github.com/eclipse/openj9/issues/10313#issuecomment-682071756

Unfortunately, it appears that core files can't be produced on MacOS (see AdoptOpenJDK/openjdk-build#1778). Without a core file, I don't really have a way of investigating this issue.

andrewcraik commented 3 years ago

I think this has to move to the backlog until the adopt issue is fixed

andrewcraik commented 3 years ago

@M-Davies do we have any corefiles to debug as yet? If we don't have them then this will need to move to backlog until we can get diagnostic files since attempts at reproduction have not been successful to the best of my understaning.

M-Davies commented 3 years ago

@M-Davies do we have any corefiles to debug as yet? If we don't have them then this will need to move to backlog until we can get diagnostic files since attempts at reproduction have not been successful to the best of my understaning.

It's hard to pick up diagnostic files from nightly builds at the best of times and with https://github.com/AdoptOpenJDK/openjdk-build/issues/1778, it makes it near impossible. ~I'll chase up that issue and see what the situation is and post back here once it's resolved~ Looks like it's something we'll be awaiting apple to fix. See the linked issue for the full problem

andrewcraik commented 3 years ago

@DanHeidinga FYI this one is missing diagnostics and so cannot be progressed - should we backlog it or close it pending diagnostics?

DanHeidinga commented 3 years ago

@andrewcraik Is there something we can add to the code around the assert to make the conditions leading to it easier to diagnose? Given we've branched for 0.23 already, I'd be OK with releasing that kind of code into the mainline - even if there's extra overhead - with the plan to pull it back out before 0.24

andrewcraik commented 3 years ago

So it looks like it can happen if we fail to compile-time resolve the defining class of a field. Now when the field is resolved how that compile-time resolve could fail is a bit of a mystery. Looking at the code though, I don't know that the fatal assert is necessary - there is a path through the code that could be crafted to avoid the problem I think. Will ponder this tonight and look again in the morning to see if there is a safe workaround. Compile time resolution is allowed to fail in general for any reason so it shouldn't be cause to kill the VM. That being said I would expect this should be pretty rare.

pshipton commented 3 years ago

Another occurrence https://ci.eclipse.org/openj9/job/Build_JDK15_x86-64_mac_Nightly/71/ there are diagnostic files including a core in https://140-211-168-230-openstack.osuosl.org/artifactory/ci-eclipse-openj9/Build_JDK15_x86-64_mac_Nightly/71/Build_JDK15_x86-64_mac_Nightly-71-20201006-235858-diagnostics.tar.gz

22:58:50  === Output from failing command(s) repeated here ===
22:58:50  * For target support_images_jmods__create_java.management.jmod_exec:
22:58:50  Assertion failed at /Users/jenkins/workspace/Build_JDK15_x86-64_mac_Nightly/build/macosx-x86_64-server-release/vm/compiler/../compiler/compile/J9SymbolReferenceTable.cpp:872: containingClass != NULL
22:58:50  VMState: 0x00050080
22:58:50    failed to get defining class of field ref cpIndex=10 in owning method J9Method=0x17b82320
22:58:50  compiling jdk/internal/org/objectweb/asm/SymbolTable.<init>(Ljdk/internal/org/objectweb/asm/ClassWriter;)V at level: warm
andrewcraik commented 3 years ago

Analysis of the core has shown that we have a resolved filed defined on the class of the current method that is being compiled. Given that we could get the flags of the field and determine its accessibility etc we should be able to get a resolved defining class. Some how the API is giving NULL. This doesn't make much sense - the answers should be consistent. That being said, this code does need to account for the defining class to not be returned. The short term tactical fix for 0.23 ONLY will be to convert the fatal assert into a compile abort. We will then continue the investigation to find out why this is inconsistent and what the JIT can do to gracefully recover. FYI @pshipton - many thanks to @Leonardo2718 @liqunl and @dsouzai for their investigation and discussion.

Leonardo2718 commented 3 years ago

10848 provides a temporary fix for the 0.23 release. Since we still need to figure what's really going in, this issue should pushed back to the next release.

pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK11_x86-64_mac_Nightly/558

Diagnostics https://140-211-168-230-openstack.osuosl.org/artifactory/ci-eclipse-openj9/Build_JDK11_x86-64_mac_Nightly/558/Build_JDK11_x86-64_mac_Nightly-558-20201118-221726-diagnostics.tar.gz

22:16:48  Generating javadoc for openj9 source files
22:16:54  Assertion failed at /Users/jenkins/workspace/Build_JDK11_x86-64_mac_Nightly/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:872: containingClass != NULL
22:16:54  VMState: 0x00050080
22:16:54    failed to get defining class of field ref cpIndex=65 in owning method J9Method=0x140744f0
22:16:54  compiling jdk/internal/org/objectweb/asm/MethodWriter.<init>(Ljdk/internal/org/objectweb/asm/ClassWriter;ILjava/lang/String;Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;I)V at level: warm
22:16:54
keithc-ca commented 3 years ago

A similar failure in https://ci.eclipse.org/openj9/job/Build_JDK15_x86-64_mac_Personal/64:

10:59:59  Assertion failed at /Users/jenkins/workspace/Build_JDK15_x86-64_mac_Personal/build/macosx-x86_64-server-release/vm/compiler/../compiler/compile/J9SymbolReferenceTable.cpp:872: containingClass != NULL
10:59:59  VMState: 0x00050080
10:59:59    failed to get defining class of field ref cpIndex=24 in owning method J9Method=0x14c8b2d0
10:59:59  compiling java/util/WeakHashMap.<init>(IF)V at level: warm
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK15_ppc64_aix_Nightly/118

22:50:28  Assertion failed at /home/jenkins/workspace/Build_JDK15_ppc64_aix_Nightly/build/aix-ppc64-server-release/vm/compiler/../compiler/compile/J9SymbolReferenceTable.cpp:872: containingClass != NULL
22:50:28  VMState: 0x00050080
22:50:28    failed to get defining class of field ref cpIndex=24 in owning method J9Method=30155fd0
22:50:28  compiling sun/nio/ch/FileChannelImpl.<init>(Ljava/io/FileDescriptor;Ljava/lang/String;ZZZLjava/lang/Object;)V at level: cold
andrew-m-leonard commented 3 years ago

New occurrence: https://ci.adoptopenjdk.net/view/Failing%20Builds/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-linux-ppc64le-openj9-linuxXL/413/console

00:11:29  Assertion failed at ../../../../../openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:872: containingClass != NULL
00:11:29  VMState: 0x00050080
00:11:29    failed to get defining class of field ref cpIndex=3 in owning method J9Method=0x100004321780
00:11:29  compiling java/util/Spliterators$IteratorSpliterator.estimateSize()J at level: cold
00:11:29  #0: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0xad6aa0) [0x100001c76aa0]
00:11:29  #1: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0xae725c) [0x100001c8725c]
00:11:29  #2: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x6f4348) [0x100001894348]
00:11:29  #3: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x6f62dc) [0x1000018962dc]
00:11:29  #4: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x182b18) [0x100001322b18]
00:11:29  #5: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x3402cc) [0x1000014e02cc]
00:11:29  #6: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x343d2c) [0x1000014e3d2c]
00:11:29  #7: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x316ee0) [0x1000014b6ee0]
00:11:29  #8: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x318afc) [0x1000014b8afc]
00:11:29  #9: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x3191d4) [0x1000014b91d4]
00:11:29  #10: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x6b95bc) [0x1000018595bc]
00:11:29  #11: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x693d78) [0x100001833d78]
00:11:29  #12: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x1aab28) [0x10000134ab28]
00:11:29  #13: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x1abddc) [0x10000134bddc]
00:11:29  #14: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9prt29.so(+0x3d508) [0x100000dcd508]
00:11:29  #15: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x1ae1ec) [0x10000134e1ec]
00:11:29  #16: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x1ae94c) [0x10000134e94c]
00:11:29  #17: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x1a8ff4) [0x100001348ff4]
00:11:29  #18: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x1a9710) [0x100001349710]
00:11:29  #19: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x1a9818) [0x100001349818]
00:11:29  #20: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9prt29.so(+0x3d508) [0x100000dcd508]
00:11:29  #21: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9jit29.so(+0x1a9e64) [0x100001349e64]
00:11:29  #22: /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/jdk/lib/default/libj9thr29.so(+0x11268) [0x100000e61268]
00:11:29  #23: /lib64/libpthread.so.0(+0x8cd4) [0x1000000c8cd4]
00:11:29  #24: function clone+0xe4 [0x100000297f14]
00:11:29  
00:11:29  JVMDUMP039I Processing dump event "abort", detail "" at 2020/11/24 00:11:28 - please wait.
00:11:29  JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/make/core.20201124.001128.24377.0001.dmp' in response to an event
00:11:38  JVMDUMP010I System dump written to /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/make/core.20201124.001128.24377.0001.dmp
00:11:38  JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/make/javacore.20201124.001128.24377.0002.txt' in response to an event
00:11:40  JVMDUMP010I Java dump written to /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/make/javacore.20201124.001128.24377.0002.txt
00:11:40  JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/make/Snap.20201124.001128.24377.0003.trc' in response to an event
00:11:40  JVMDUMP010I Snap dump written to /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/make/Snap.20201124.001128.24377.0003.trc
00:11:40  JVMDUMP032I JVM requested JIT dump using '/home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/make/jitdump.20201124.001128.24377.0004.dmp' in response to an event
00:11:41  JVMDUMP010I JIT dump written to /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/make/jitdump.20201124.001128.24377.0004.dmp
00:11:41  JVMDUMP013I Processed dump event "abort", detail "".
00:11:42  CreateJmods.gmk:217: recipe for target '/home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/images/jmods/java.instrument.jmod' failed
00:11:42  gmake[3]: *** [/home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-ppc64le-openj9-linuxXL/workspace/build/src/build/linux-ppc64le-normal-server-release/images/jmods/java.instrument.jmod] Error 1
00:11:42  make/Main.gmk:357: recipe for target 'java.instrument-jmod' failed
00:11:42  gmake[2]: *** [java.instrument-jmod] Error 1
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK11_x86-64_mac_OMR/772

05:24:09  Generating javadoc for openj9 source files
05:24:13  Assertion failed at /Users/jenkins/workspace/Build_JDK11_x86-64_mac_OMR/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:872: containingClass != NULL
05:24:13  VMState: 0x000501ff
05:24:13    failed to get defining class of field ref cpIndex=4 in owning method J9Method=0x10782950
05:24:13  compiling jdk/internal/org/objectweb/asm/MethodWriter.visitVarInsn(II)V at level: warm
liqunl commented 3 years ago

Looked at all recent core dumps, all failed compilation are AOT compiles during start up, so no-SVM AOT is used. In all failed compiles, there are field load/store of the same class before the one that's causing the crash, and the defining class of the field is the class of outer most method, which means that validation for the class should succeed.

Looking at the code In fieldAttributes, we call storeValidationRecordIfNecessary in non-SVM AOT compile https://github.com/eclipse/openj9/blob/8c76311cdcccaa6def680d362df418eca6ae9325/runtime/compiler/env/j9method.cpp#L1845

And storeValidationRecordIfNecessary calls to TR_ResolvedJ9Method::definingClassFromCPFieldRef to get defining class https://github.com/eclipse/openj9/blob/8c76311cdcccaa6def680d362df418eca6ae9325/runtime/compiler/env/j9method.cpp#L1715

Since fieldAttributes returned true (resolved is true), that means we were able to get defining class and validate the field.

But when we call TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef, which calls to storeValidationRecordIfNecessary with the same parameters used by fieldAttributes, we get NULL. Either TR_ResolvedJ9Method::definingClassFromCPFieldRef returned NULL or storeValidationRecordIfNecessary failed https://github.com/eclipse/openj9/blob/8c76311cdcccaa6def680d362df418eca6ae9325/runtime/compiler/env/j9method.cpp#L2022-L2040

Either way, the result is inconsistent. We have to find out what is inconsistent, and proceed from there.

@Leonardo2718 There's not much I can find from the core dump. If we can't reproduce it locally, I suggest we add more information to the assert, and wait until next crash. The extra information to print can be the result of TR_ResolvedJ9Method::definingClassFromCPFieldRef from TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef

Leonardo2718 commented 3 years ago

I'm in the process of trying to reproduce this myself. While that's happening, I figured we might as well try adding the extra info to the assert message, so I've opened #11279.

pshipton commented 3 years ago

See https://github.com/eclipse/openj9/issues/11313

00:24:06  Assertion failed at /Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:879: containingClass != NULL
00:24:06  VMState: 0x000501ff
00:24:06    failed to get defining class of field ref cpIndex=11 in owning method J9Method=0xa0f5010
00:24:06    TR_ResolvedJ9Method::definingClassFromCPFieldRef=0xa10d900, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=0xa10d900
00:24:06    TR_UseSymbolValidationManager=0
00:24:06  compiling java/nio/file/spi/FileSystemProvider.newInputStream(Ljava/nio/file/Path;[Ljava/nio/file/OpenOption;)Ljava/io/InputStream; at level: warm
00:24:06  
00:24:06  JVMDUMP039I Processing dump event "abort", detail "" at 2020/12/01 00:24:05 - please wait.
00:24:06  JVMDUMP032I JVM requested System dump using '/Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9/workspace/build/src/make/core.20201201.002405.10028.0001.dmp' in response to an event

https://ci.adoptopenjdk.net/view/Failing%20Builds/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-mac-x64-openj9/831/console

liqunl commented 3 years ago

The latest crash shows that we got different results calling TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef. @dsouzai When can this happen? Can we switch from SVM to non-SVM in a compilation? Can we first fail remembering a class but later succeed?

dsouzai commented 3 years ago

When can this happen?

No idea, this doesn't make sense. I've been talking to @Leonardo2718 offline, and I can't find come up with even the craziest scenario for how this could happen.

In order to enter the path where the assert exists, we must've returned true for owningMethod->fieldAttributes(...). TR_ResolvedRelocatableJ9Method::fieldAttributes calls TR_ResolvedRelocatableJ9Method::storeValidationRecordIfNecessary which would've invoked TR_ResolvedJ9Method::definingClassFromCPFieldRef. It also calls rememberClass which would've succeeded because 1. fieldAttributes returned true, and 2. I'm told the defining class is the class of the method we're compiling and so we would've already remembered it at the start of the compile...though even if it wasn't, the fact that fieldAttributes returned true means that we succeeded in remembering the class. fieldAttributes also does create an instance field validation record, but it must've succeeded nonetheless.

So, when we call TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef:

TR_OpaqueClassBlock *
TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef(
   TR::Compilation *comp,
   I_32 cpIndex,
   bool isStatic)
   {
   TR_OpaqueClassBlock *clazz = TR_ResolvedJ9Method::definingClassFromCPFieldRef(comp, cp(), cpIndex, isStatic);

   bool valid = false;
   if (comp->getOption(TR_UseSymbolValidationManager))
      valid = comp->getSymbolValidationManager()->addDefiningClassFromCPRecord(clazz , cp(), cpIndex, isStatic);
   else
      valid = storeValidationRecordIfNecessary(comp, cp(), cpIndex, isStatic ? TR_ValidateStaticField : TR_ValidateInstanceField, ramMethod());

   if (!valid)
      clazz = NULL;

   return clazz;
   }

TR_ResolvedJ9Method::definingClassFromCPFieldRef should've returned a non-NULL class, because it succeeded earlier, and storeValidationRecordIfNecessary(comp, cp(), cpIndex, isStatic ? TR_ValidateStaticField : TR_ValidateInstanceField, ramMethod()) should've returned true, because we would've literally done the same call before.

You then add on the fact that with Leo's assert, we make the same call to TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef again which now returns a non-NULL class and it makes things even more confusing.

Can we switch from SVM to non-SVM in a compilation?

You'd have to run with export TR_DontDisableSVMDuringStartup=1

Can we first fail remembering a class but later succeed?

While the code allows it, I don't know if there's any valid reason for while this could happen. If a class failed to be remembered, it's because its J9ROMClass doesn't exist in the SCC, or the SCC is full. In this case though, if the class we're trying to remember is the class of the method we're compiling; we would've already remembered it at the start of the compile, or we wouldn't have done the compile to begin with.

liqunl commented 3 years ago

I wonder if TR_ResolvedJ9Method::definingClassFromCPFieldRef returns different results, or it's just that we failed storing the record. We can add a version of TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef that gives us the result of TR_ResolvedJ9Method::definingClassFromCPFieldRef during its execution. And dump the information in our assert. If the assert fires and we get TR_ResolvedJ9Method::definingClassFromCPFieldRef returns non-null, then it's a problem in AOT record creation. @Leonardo2718

To fix it in the coming release, I think we can get fieldAttributes to return the defining class for us, then we should always get the defining class if fieldAttributes says the field is resolved.

0xdaryl commented 3 years ago

For 0.24, why don't we fail the compilation as we did in the 0.23 release (@Leonardo2718 to reintroduce the fix in 0.24) to "gracefully" avoid this problem. Until we have more data on what the actual problem might be I think that is the safest way to work around this problem.

In the meantime, in the head stream, I like @liqunl's suggestion of capturing more of the API results that are producing inconsistent behaviour and including them in the assert so that we can hopefully reason as to what the issue might be.

Leonardo2718 commented 3 years ago

Opened #11431 to "gracefully avoid this problem" in 0.24.

pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK11_ppc64_aix_OMR/805

Diagnostics https://140-211-168-230-openstack.osuosl.org/artifactory/ci-eclipse-openj9/Build_JDK11_ppc64_aix_OMR/805/Build_JDK11_ppc64_aix_OMR-805-20201211-075646-diagnostics.tar.gz

07:55:14  Creating images/jmods/jdk.jlink.jmod
07:55:14  Creating images/jmods/jdk.jshell.jmod
07:55:14  Creating images/jmods/jdk.jsobject.jmod
07:55:15  Creating images/jmods/jdk.localedata.jmod
07:55:15  Creating images/jmods/jdk.management.agent.jmod
07:55:15  Creating images/jmods/jdk.management.jmod
07:55:22  Creating images/jmods/jdk.naming.dns.jmod
07:56:02  Assertion failed at /home/jenkins/workspace/Build_JDK11_ppc64_aix_OMR/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:879: containingClass != NULL
07:56:02  VMState: 0x00050080
07:56:02    failed to get defining class of field ref cpIndex=10 in owning method J9Method=301ffa78
07:56:02    TR_ResolvedJ9Method::definingClassFromCPFieldRef=301fdf00, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=0
07:56:02    TR_UseSymbolValidationManager=0
07:56:02  compiling java/util/TimerThread.mainLoop()V at level: cold
07:56:02  TR::fatal_assertion(const char*,int,const char*,const char*,...)+0xfc
07:56:02  J9::SymbolReferenceTable::findOrCreateShadowSymbol(TR::ResolvedMethodSymbol*,int,bool)+0xa90
07:56:02  TR_J9ByteCodeIlGenerator::loadInstance(int)+0x2f8
07:56:02  TR_J9ByteCodeIlGenerator::walker(TR::Block*)+0x7564
07:56:02  TR_J9ByteCodeIlGenerator::genILFromByteCodes()+0x3074
07:56:02  TR_J9ByteCodeIlGenerator::internalGenIL()+0xa08
07:56:02  TR_J9ByteCodeIlGenerator::genIL()+0x37c
07:56:02  OMR::ResolvedMethodSymbol::genIL(TR_FrontEnd*,TR::Compilation*,TR::SymbolReferenceTable*,TR::IlGenRequest&)+0xe88
07:56:02  OMR::Compilation::compile()+0x2198
07:56:02  TR::CompilationInfoPerThreadBase::compile(J9VMThread*,TR::Compilation*,TR_ResolvedMethod*,TR_J9VMBase&,TR_OptimizationPlan*,const TR::SegmentAllocator&)+0x1054
07:56:02  TR::CompilationInfoPerThreadBase::wrappedCompile(J9PortLibrary*,void*)+0x2194
07:56:02  omrsig_protect+0x614
07:56:02  TR::CompilationInfoPerThreadBase::compile(J9VMThread*,TR_MethodToBeCompiled*,J9::J9SegmentProvider&)+0x868
07:56:02  TR::CompilationInfoPerThread::processEntry(TR_MethodToBeCompiled&,J9::J9SegmentProvider&)+0x9e8
07:56:02  TR::CompilationInfoPerThread::processEntries()+0x840
07:56:02  protectedCompilationThreadProc(J9PortLibrary*,TR::CompilationInfoPerThread*)+0x544
07:56:02  omrsig_protect+0x614
07:56:02  compilationThreadProc(void*)+0x644
07:56:02  thread_wrapper+0x73c
07:56:02  _pthread_body+0x128
07:56:02  +0x0
0xdaryl commented 3 years ago

This issue continues to be worked in the master branch to get to the root cause of the problem. Additional asserts are being added to help provide more data points to understand this unusual situation.

In the meantime, a workaround solution was added to the 0.24 release (#11431) to gracefully fail the compilation when this incongruous state is detected. This is the same solution employed in the 0.23 release.

This issue will be moved forward to the 0.25 release as the investigation continues.

pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK11_ppc64_aix_Nightly/595

22:54:48  Assertion failed at /home/jenkins/workspace/Build_JDK11_ppc64_aix_Nightly/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
22:54:48  VMState: 0x00050080
22:54:48    failed to get defining class of field ref cpIndex=12 in owning method J9Method=30147b00
22:54:48    TR_ResolvedJ9Method::definingClassFromCPFieldRef=30148000, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=30148000
22:54:48    fromResolvedJ9Method=30148000
22:54:48    TR_UseSymbolValidationManager=0
22:54:48  compiling sun/nio/ch/FileChannelImpl.size()J at level: cold
22:54:48  TR::fatal_assertion(const char*,int,const char*,const char*,...)+0xfc
22:54:48  J9::SymbolReferenceTable::findOrCreateShadowSymbol(TR::ResolvedMethodSymbol*,int,bool)+0xaa0
22:54:48  TR_J9ByteCodeIlGenerator::loadInstance(int)+0x2f8
22:54:48  TR_J9ByteCodeIlGenerator::walker(TR::Block*)+0x7564
22:54:48  TR_J9ByteCodeIlGenerator::genExceptionHandlers(TR::Block*)+0x1d34
22:54:48  TR_J9ByteCodeIlGenerator::genILFromByteCodes()+0x3074
22:54:48  TR_J9ByteCodeIlGenerator::internalGenIL()+0xa08
22:54:48  TR_J9ByteCodeIlGenerator::genIL()+0x37c
22:54:48  OMR::ResolvedMethodSymbol::genIL(TR_FrontEnd*,TR::Compilation*,TR::SymbolReferenceTable*,TR::IlGenRequest&)+0xe88
22:54:48  OMR::Compilation::compile()+0x2198
22:54:48  TR::CompilationInfoPerThreadBase::compile(J9VMThread*,TR::Compilation*,TR_ResolvedMethod*,TR_J9VMBase&,TR_OptimizationPlan*,const TR::SegmentAllocator&)+0x1054
22:54:48  TR::CompilationInfoPerThreadBase::wrappedCompile(J9PortLibrary*,void*)+0x2194
22:54:48  omrsig_protect+0x614
22:54:48  TR::CompilationInfoPerThreadBase::compile(J9VMThread*,TR_MethodToBeCompiled*,J9::J9SegmentProvider&)+0x868
22:54:48  TR::CompilationInfoPerThread::processEntry(TR_MethodToBeCompiled&,J9::J9SegmentProvider&)+0x9e8
22:54:48  TR::CompilationInfoPerThread::processEntries()+0x840
22:54:48  protectedCompilationThreadProc(J9PortLibrary*,TR::CompilationInfoPerThread*)+0x544
22:54:48  omrsig_protect+0x614
22:54:48  compilationThreadProc(void*)+0x644
22:54:48  thread_wrapper+0x73c
22:54:48  _pthread_body+0x128
22:54:48  +0x0
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK15_x86-64_mac_Nightly/142

22:45:54  Assertion failed at /Users/jenkins/workspace/Build_JDK15_x86-64_mac_Nightly/build/macosx-x86_64-server-release/vm/compiler/../compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
22:45:54  VMState: 0x000501ff
22:45:54    failed to get defining class of field ref cpIndex=9 in owning method J9Method=0x68d60f8
22:45:54    TR_ResolvedJ9Method::definingClassFromCPFieldRef=0x68d3700, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=0x68d3700
22:45:54    fromResolvedJ9Method=0x68d3700
22:45:54    TR_UseSymbolValidationManager=0
22:45:54  compiling jdk/internal/module/ModuleInfo$ConstantPool.<init>(Ljava/io/DataInput;)V at level: warm
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDKnext_ppc64_aix_OpenJDK/286 There is ~500MB of JIT output.

01:48:03  Assertion failed at /home/jenkins/workspace/Build_JDKnext_ppc64_aix_OpenJDK/build/aix-ppc64-server-release/vm/compiler/../compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
01:48:03  VMState: 0x00050080
01:48:03    failed to get defining class of field ref cpIndex=57 in owning method J9Method=30228720
01:48:03    TR_ResolvedJ9Method::definingClassFromCPFieldRef=30225d00, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=30225d00
01:48:03    fromResolvedJ9Method=30225d00
01:48:03    TR_UseSymbolValidationManager=0
01:48:03  compiling java/util/regex/Pattern.compile()V at level: cold
01:48:03  Unhandled exception
01:48:03  Type=Segmentation error vmState=0x00050080
01:48:03  J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000032
01:48:03  Handler1=09001000A1335948 Handler2=09001000A1309980
01:48:03  R0=0000000000000000 R1=00000100206319F0 R2=08001000A01DC198 R3=0000010020631A60
01:48:03  R4=09000000167DFF10 R5=7CC802A6F8010010 R6=08001000A00371A0 R7=0000000000260000
01:48:03  R8=0000000000000000 R9=5061747465726E2E R10=0000000000000001 R11=000001002063A370
01:48:03  R12=000000004220442B R13=0000010020645800 R14=0000010020635520 R15=0000010020635408
01:48:03  grep_64: write error: Broken pipe
01:48:03     ... (rest of output omitted)
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK11_x86-64_mac_Nightly/608

22:41:06  Assertion failed at /Users/jenkins/workspace/Build_JDK11_x86-64_mac_Nightly/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
22:41:06  VMState: 0x000501ff
22:41:06    failed to get defining class of field ref cpIndex=7 in owning method J9Method=0x12cd09f8
22:41:06    TR_ResolvedJ9Method::definingClassFromCPFieldRef=0x12ccdf00, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=0x12ccdf00
22:41:06    fromResolvedJ9Method=0x12ccdf00
22:41:06    TR_UseSymbolValidationManager=0
22:41:06  compiling jdk/internal/module/ModuleInfo$ConstantPool.<init>(Ljava/io/DataInput;)V at level: warm
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK16_ppc64_aix_OpenJDK16/5

18:40:11  Assertion failed at /home/jenkins/workspace/Build_JDK16_ppc64_aix_OpenJDK16/build/aix-ppc64-server-release/vm/compiler/../compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
18:40:11  VMState: 0x00050080
18:40:11    failed to get defining class of field ref cpIndex=12 in owning method J9Method=301599e0
18:40:11    TR_ResolvedJ9Method::definingClassFromCPFieldRef=30159e00, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=30159e00
18:40:11    fromResolvedJ9Method=30159e00
18:40:11    TR_UseSymbolValidationManager=0
18:40:11  compiling sun/nio/fs/UnixChannelFactory.newFileChannel(ILsun/nio/fs/UnixPath;Ljava/lang/String;Ljava/util/Set;I)Ljava/nio/channels/FileChannel; at level: cold
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK15_ppc64_aix_xl_Personal/11

16:28:25  Assertion failed at /home/jenkins/workspace/Build_JDK15_ppc64_aix_xl_Personal/build/aix-ppc64-server-release/vm/compiler/../compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
16:28:25  VMState: 0x00050080
16:28:25    failed to get defining class of field ref cpIndex=84 in owning method J9Method=10021f77628
16:28:25    TR_ResolvedJ9Method::definingClassFromCPFieldRef=10021f54800, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=10021f54800
16:28:25    fromResolvedJ9Method=10021f54800
16:28:25    TR_UseSymbolValidationManager=0
16:28:25  compiling java/util/HashMap.values()Ljava/util/Collection; at level: warm
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK11_ppc64_aix_Personal/852

21:14:57  Assertion failed at /home/jenkins/workspace/Build_JDK11_ppc64_aix_Personal/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
21:14:57  VMState: 0x00050080
21:14:57    failed to get defining class of field ref cpIndex=14 in owning method J9Method=301e2118
21:14:57    TR_ResolvedJ9Method::definingClassFromCPFieldRef=301e1f00, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=301e1f00
21:14:57    fromResolvedJ9Method=301e1f00
21:14:57    TR_UseSymbolValidationManager=0
21:14:57  compiling openj9/internal/tools/attach/target/FileLock.lockFile(Z)Z at level: cold
21:14:57  TR::fatal_assertion(const char*,int,const char*,const char*,...)+0xfc
21:14:57  J9::SymbolReferenceTable::findOrCreateShadowSymbol(TR::ResolvedMethodSymbol*,int,bool)+0xaa0
21:14:57  TR_J9ByteCodeIlGenerator::storeInstance(int)+0x2f8
21:14:57  TR_J9ByteCodeIlGenerator::walker(TR::Block*)+0x7564
21:14:57  TR_J9ByteCodeIlGenerator::genILFromByteCodes()+0x3074
21:14:57  TR_J9ByteCodeIlGenerator::internalGenIL()+0xa08
21:14:57  TR_J9ByteCodeIlGenerator::genIL()+0x37c
21:14:57  OMR::ResolvedMethodSymbol::genIL(TR_FrontEnd*,TR::Compilation*,TR::SymbolReferenceTable*,TR::IlGenRequest&)+0xe88
21:14:57  OMR::Compilation::compile()+0x2198
21:14:57  TR::CompilationInfoPerThreadBase::compile(J9VMThread*,TR::Compilation*,TR_ResolvedMethod*,TR_J9VMBase&,TR_OptimizationPlan*,const TR::SegmentAllocator&)+0x1054
21:14:57  TR::CompilationInfoPerThreadBase::wrappedCompile(J9PortLibrary*,void*)+0x21b0
21:14:57  omrsig_protect+0x614
21:14:57  TR::CompilationInfoPerThreadBase::compile(J9VMThread*,TR_MethodToBeCompiled*,J9::J9SegmentProvider&)+0x868
21:14:57  TR::CompilationInfoPerThread::processEntry(TR_MethodToBeCompiled&,J9::J9SegmentProvider&)+0x9e8
21:14:57  TR::CompilationInfoPerThread::processEntries()+0x840
21:14:57  protectedCompilationThreadProc(J9PortLibrary*,TR::CompilationInfoPerThread*)+0x544
21:14:57  omrsig_protect+0x614
21:14:57  compilationThreadProc(void*)+0x644
21:14:57  thread_wrapper+0x73c
21:14:57  _pthread_body+0x128
21:14:57  +0x0
dsouzai commented 3 years ago

Several of us (Mark, Leo, Andrew, Liqun, myself) have looked at this code and we haven't been able figure out how the call to TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef can return NULL one time and non-NULL a second time, given that both fieldAttributes and TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef both call storeValidationRecordIfNecessary with the same parameters. In fact, there's been an example [1] where the third time it returns NULL again. We're starting to consider the possibility that there's some timing related issue related to the SCC APIs. However, to verify this, @Leonardo2718 is coding up some changes to provide a trace of calls made in J9::SymbolReferenceTable::findOrCreateShadowSymbol starting from fieldAttributes up to the point where we assert.

@hangshao0 do you know if there's any kind of sanity checking or other tracing in the SCC code that might be helpful here?

[1] https://github.com/eclipse/openj9/issues/9416#issuecomment-743283919

hangshao0 commented 3 years ago

We're starting to consider the possibility that there's some timing related issue related to the SCC APIs

I guess the APIs are findSharedData()/storeSharedData(). Probably worth turning on -Xshareclasses sub-option verboseData, and the trace points in SH_CacheMap::findSharedData()/SH_CacheMap::storeSharedData().

pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK11_x86-64_mac_mixed_Nightly/3

22:18:52  Assertion failed at /Users/jenkins/workspace/Build_JDK11_x86-64_mac_mixed_Nightly/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
22:18:52  VMState: 0x00050080
22:18:52    failed to get defining class of field ref cpIndex=51 in owning method J9Method=0xf67cef0
22:18:52    TR_ResolvedJ9Method::definingClassFromCPFieldRef=0xf67d600, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=0xf67d600
22:18:52    fromResolvedJ9Method=0xf67d600
22:18:52    TR_UseSymbolValidationManager=0
22:18:52  compiling jdk/internal/org/objectweb/asm/MethodWriter.<init>(Ljdk/internal/org/objectweb/asm/ClassWriter;ILjava/lang/String;Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;I)V at level: warm
pshipton commented 3 years ago

https://ci.eclipse.org/openj9/job/Build_JDK11_x86-64_mac_OMR/820

16:28:02  Assertion failed at /Users/jenkins/workspace/Build_JDK11_x86-64_mac_OMR/openj9/runtime/compiler/compile/J9SymbolReferenceTable.cpp:882: containingClass != NULL
16:28:02  VMState: 0x00050080
16:28:02    failed to get defining class of field ref cpIndex=47 in owning method J9Method=0xb221ff0
16:28:02    TR_ResolvedJ9Method::definingClassFromCPFieldRef=0xb21c600, TR_ResolvedRelocatableJ9Method::definingClassFromCPFieldRef=0xb21c600
16:28:02    fromResolvedJ9Method=0xb21c600
16:28:02    TR_UseSymbolValidationManager=0
16:28:02  compiling jdk/internal/org/objectweb/asm/MethodWriter.<init>(Ljdk/internal/org/objectweb/asm/ClassWriter;ILjava/lang/String;Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;I)V at level: warm