JDK11 s390x linux jit build failed due to "Illegal instruction exception"

[cjjdespres]

Observed internally in the JITServer nightly test pipeline, in this build: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK11_s390x_linux_jit_Personal/1524/. Note that the compiler is not built with -XX:+UseJITServer as far as I am aware - only the subsequent tests are run with that option - so this isn't JITServer-specific.

Console log:

03:14:09  Creating support/modules_cmds/jdk.jlink/jmod from 1 file(s)
03:14:10  Unhandled exception
03:14:10  Type=Illegal instruction vmState=0x00000000
03:14:10  J9Generic_Signal_Number=00000048 Signal_Number=00000004 Error_Value=00000000 Signal_Code=00000001
03:14:10  Handler1=00000200252C8F50 Handler2=00000200255B1A60
03:14:10  gpr0=0000000000000019 gpr1=000000008F2369F0 gpr2=0000000000000000 gpr3=0000000000069800
03:14:10  gpr4=0000020036BAB902 gpr5=00000200281E9AD8 gpr6=000000008F236920 gpr7=00000200368F4A00
03:14:10  gpr8=000000000009D200 gpr9=0101010000010101 gpr10=000000008F2369F0 gpr11=0000010100010000
03:14:10  gpr12=00000200249EBF98 gpr13=0000000000069800 gpr14=000000000006A3C8 gpr15=00000200249EBC18
03:14:10  psw=0000020036BAB7E8 mask=0705100180000000 fpc=00080000 bea=0000020036BAB91A
03:14:10  fpr0=000002002671ddc8 (f: 644996544.000000, d: 1.086781e-311)
03:14:10  fpr1=4bf77dd900000000 (f: 0.000000, d: 9.216195e+57)
03:14:10  fpr2=0000000000000000 (f: 0.000000, d: 0.000000e+00)
03:14:10  fpr3=3886bdd320202020 (f: 538976256.000000, d: 2.138601e-36)
03:14:10  fpr4=4bf77dd9b2000000 (f: 2986344448.000000, d: 9.216199e+57)
03:14:10  fpr5=3e924925a0c00000 (f: 2696937472.000000, d: 2.724786e-07)
03:14:10  fpr6=0000000000000000 (f: 0.000000, d: 0.000000e+00)
03:14:10  fpr7=3e3a332500000000 (f: 0.000000, d: 6.100112e-09)
03:14:10  fpr8=00000200281e9938 (f: 673093952.000000, d: 1.086794e-311)
03:14:10  fpr9=0f0e0d0c0b0a0920 (f: 185207072.000000, d: 3.691916e-236)
03:14:10  fpr10=0000000038cdf9b8 (f: 953022912.000000, d: 4.708559e-315)
03:14:10  fpr11=00000000ccaa009e (f: 3433693440.000000, d: 1.696470e-314)
03:14:10  fpr12=0000000000063008 (f: 405512.000000, d: 2.003495e-318)
03:14:10  fpr13=0000000000000000 (f: 0.000000, d: 0.000000e+00)
03:14:10  fpr14=000003ffd14782fc (f: 3511124736.000000, d: 2.172536e-311)
03:14:10  fpr15=0000000000000000 (f: 0.000000, d: 0.000000e+00)
03:14:10  
03:14:10  Compiled_method=java/lang/invoke/AbstractValidatingLambdaMetafactory.<init>(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/invoke/MethodType;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;Z[Ljava/lang/Class;[Ljava/lang/invoke/MethodType;)V
03:14:10  Target=2_90_20240802_1032 (Linux 4.18.0-553.16.1.el8_10.s390x)
03:14:10  CPU=s390x (4 logical CPUs) (0x1f5b98000 RAM)
03:14:10  ----------- Stack Backtrace -----------
03:14:10   (0x0000020036BAB7E8 [<unknown>+0x0])

This may have happened in an earlier build as well. There was no error message in that run, other than that the build failed, but the logs appear to show that a core dump was generated as a result of the failure:

[2024-08-09T07:15:25.437Z] + find . -name 'core.*.dmp' -o -name 'javacore.*.txt' -o -name 'jitdump.*.dmp' -o -name 'Snap.*.trc' -o -name '*.debuginfo' -o -type d -name ddr_info -prune -o -type d -path ./build/linux-s390x-normal-server-release/make-support/failure-logs -prune -o -type d -path ./build/linux-s390x-normal-server-release/jdk -prune -o -type d -path ./build/linux-s390x-normal-server-release/support/modules_libs -prune
[2024-08-09T07:15:25.437Z] + sed 's#^./##'
[2024-08-09T07:15:25.437Z] + tar -zcvf Build_JDK11_s390x_linux_jit_Personal-1513-20240809-071521-diagnostics.tar.gz -T -
[2024-08-09T07:15:25.849Z] make/core.20240809.065808.5102.0001.dmp
[2024-08-09T07:16:15.828Z] make/javacore.20240809.065808.5102.0002.txt
[2024-08-09T07:16:15.828Z] make/Snap.20240809.065808.5102.0003.trc
[2024-08-09T07:16:15.828Z] make/jitdump.20240809.065808.5102.0004.dmp

Possibly related is the infra issue https://github.ibm.com/runtimes/infrastructure/issues/9763. That has a different error (build failure due to "connection refused") but perhaps the underlying cause is the same.

These two failing builds occurred on rtj-rhel8s390x-svl-build-1.svl.ibm.com and rtj-rhel8s390x-svl-build-2.svl.ibm.com. I think the builds in the linked infra issue also all occurred on one of these two machines, but I haven't checked exhaustively.

[cjjdespres]

Another instance of the same illegal instruction exception: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK11_s390x_linux_jit_Personal/1526/.

[pshipton]

@r30shah is this a VICOM issue, or something else?