SIGSEGV 1.8.0_352-b08

[montanafoertsch-toast]

Describe the bug

I'm running into a segmentation fault running with the Corretto 8 JRE on my mac (Monterey) and on a machine in our build pipeline running linux.

I am more than willing to hear that this isn't a Corretto specific issue and should be submitted elsewhere. I'm not very experienced debugging at this level and may be missing important/obvious(to more experienced folks) signs from the crash file.

hr_err file attached, but a snippet from the top for this ticket description.

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000000000000000, pid=8423, tid=0x00007faf6f2ff700
#
# JRE version: OpenJDK Runtime Environment (8.0_352-b08) (build 1.8.0_352-b08)
# Java VM: OpenJDK 64-Bit Server VM (25.352-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  0x0000000000000000

• This crash does not occur with Corretto 11 but I can't use 11 for reasons outside of my control.
• This crash also occurs on 8.322.06.2

  To Reproduce

• I won't be able to submit repro steps as this is coming from running a large batch of integration tests for a webservice through gradle and trying to remove components seemed to point more to a slimmer application being less likely to trigger the failure rather than a removed component pointing directly to the cause.

Expected behavior

No SIGSEGV

Platform information

System: Linux 5.10.102-99.473.amzn2.x86_64 x86_64 Java -version: openjdk version "1.8.0_352" OpenJDK Runtime Environment Corretto-8.352.08.1 (build 1.8.0_352-b08) OpenJDK 64-Bit Server VM Corretto-8.352.08.1 (build 25.352-b08, mixed mode)

Additional context

jstack -m against the produced core.#### file reported: ----------------- 8429 ----------------- 0x00007faf6e52dca0 __GI_raise + 0x110 0x00007faf6dff2938 _ZN7VMError14report_and_dieEv + 0x5d8 0x00007faf6de39855 JVM_handle_linux_signal + 0x1a5 0x00007faf6de2cdf8 _Z13signalHandleriP9siginfo_tPv + 0x38 hs_err.txt

[navyxliu]

hi, @montanafoertsch-toast ,

jstack can't be used to parse core files. you need to use clhsdb or GUI version hsdb.

I read your hs_err.txt. it's from AL2 on AMD EPYC. Are you saying that you can trigger this problem on both AL2 and MacOS? Is there a reliable way to trigger this issue?

The direct reason that hotspot gets crashed because your pc is 0. rsp[0] is 0x00007faf5b919684, it points to nothing. It's not clear what error leads to this error.

Besides HotSpot, it's also possible to have an error in JNI code. I see that you are using 'libcirce-checksum.so'. do you know what's it? better off check with '-Xcheck:jni'.

[montanafoertsch-toast]

@navyxliu +1 thanks for the pointers, I'll see what I can dig up

• I get the SIGSEGV on both AL2 and MacOS, but haven't been able to get around some of the security entitlements on Monterey to enable a core dump to be written, so I can't say 100% the crash appears to be caused by the same issue. Still mucking around with settings trying to generate this one
  • As for reliability, I can generate it about 95% of the time running a suite of integration tests but that spins up quite a few things. I've been trying to pare this back a bit to narrow it down but having trouble doing so with how interconnected parts of the app are.
• libcirce-checksum looks like its from apache bookkeeper which I think is being used by a pulsar client dependency. Will try '-Xcheck:jni'

[montanafoertsch-toast]

@navyxliu sorry for the delay, managed to get some more free time.

• running with '-Xcheck:jni' turns up a number of warnings of the types:
  • WARNING: JNI local refs: N, exceeds capacity: N-1
  • WARNING in native method: JNI call made without checking exceptions when required to from CallFooMethod Void/Int/Object/StaticObject/etc.
• no JNI warnings are printed near the time of the crash (not sure if they would be or if the jvm terminating would prevent it?)

I'll muck about with clhsdb some more tomorrow and see what I can find (currently getting ERROR: failed to workaround classshareing when trying it on core dumps from my mac)