jdk11/15 intermittent Segmentation fault: 11

[andrew-m-leonard]

We are intermittently seeing a Segmentation fault: 11 https://ci.adoptopenjdk.net/view/Failing%20Builds/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-mac-x64-openj9-macosXL/771/consoleFull build-macstadium-macos1014-x64-2

10:18:41  Creating support/modules_libs/java.base/jli/libjli.dylib from 8 file(s)
10:18:41  Creating support/modules_libs/java.base/libnet.dylib from 22 file(s)
10:18:41  Creating support/modules_libs/java.base/libnio.dylib from 23 file(s)
10:18:41  Creating support/modules_libs/java.base/libosxsecurity.dylib from 1 file(s)
10:18:52  Compiling 2739 files for java.desktop
10:18:52  Updating support/src.zip
10:18:56  make[3]: *** [/Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9-macosXL/workspace/build/src/build/macosx-x86_64-normal-server-release/support/modules_libs/java.base/libverify.dylib] Segmentation fault: 11
10:18:56  make[3]: *** Deleting file `/Users/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-mac-x64-openj9-macosXL/workspace/build/src/build/macosx-x86_64-normal-server-release/support/modules_libs/java.base/libverify.dylib'
10:18:56  make[3]: *** Waiting for unfinished jobs....
10:18:57  make[2]: *** [java.base-libs] Error 2
10:18:57  make[2]: *** Waiting for unfinished jobs....
10:21:21  
10:21:21  ERROR: Build failed for targets 'product-images mac-legacy-jre-bundle test-image debug-image' in configuration 'macosx-x86_64-normal-server-release' (exit code 2) 
10:21:21  Stopping sjavac server

No further details...

[andrew-m-leonard]

jdk15 also: https://ci.adoptopenjdk.net/view/Failing%20Builds/job/build-scripts/job/jobs/job/jdk15u/job/jdk15u-mac-x64-openj9-macosXL/119/console build-macstadium-macos1014-x64-2

[pshipton]

First occurrence I found for jdk15 https://ci.adoptopenjdk.net/view/Failing%20Builds/job/build-scripts/job/jobs/job/jdk15u/job/jdk15u-mac-x64-openj9-macosXL/116 build-macstadium-macos1014-x64-2 [2020-11-25T03:50:26.689Z] Source version info: [2020-11-25T03:50:26.689Z] openjdk - d3b60198f1 [2020-11-25T03:50:26.689Z] openj9 - 8c76311cd [2020-11-25T03:50:26.689Z] omr - 720d59d81

Previous build, which compiled (build-macstadium-macos1014-x64-1). [2020-11-24T21:36:21.921Z] Source version info: [2020-11-24T21:36:21.921Z] openjdk - d3b60198f1 [2020-11-24T21:36:21.921Z] openj9 - 8c76311cd [2020-11-24T21:36:21.921Z] omr - 720d59d81

[pshipton]

All the Segmentation fault: 11 occur on the same machine build-macstadium-macos1014-x64-2

[pshipton]

@tajila fyi

[pshipton]

@JasonFengJ9 have you seen anything like this in internal builds? I haven't see it occur in the OpenJ9 public builds.

[JasonFengJ9]

@pshipton no, not that I can recall.

All the Segmentation fault: 11 occur on the same machine build-macstadium-macos1014-x64-2

Did this happen to hotspot builds as well?

[andrew-m-leonard]

I've not seen it on Hotspot

[pshipton]

https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-mac-x64-openj9/839 This one occurred on build-macstadium-macos1014-x64-1

[tajila]

Have we been able to capture any diagnostic files? system/java cores?

[pshipton]

There is basically no information about the Segmentation fault: 11, not much we can do atm. It's not even clear that Java is producing the Segmentation fault, as Java should produce different output and usually some diagnostic files. If a core file is created and that can be obtained and analyzed, we can determine what and where the Segmentation fault occurs.

The OpenJ9 project hasn't seen this Segmentation fault, nor internally to IBM either. It may be worth checking the versions of the tooling used on the build machines against what the OpenJ9 project is using.

[andrew-m-leonard]

Happened again on node: build-macstadium-macos1014-x64-1 I'll see if I can find any core file => nope can't find anything, although looks like on mac you need to enable native core dumps I think

[andrew-m-leonard]

Seen on Hotspot too