adoptium / aqa-tests

Home of test infrastructure for Adoptium builds
https://adoptium.net/aqavit
Apache License 2.0
129 stars 308 forks source link

April 2024 CPU - JDK21 Triage #5234

Closed smlambert closed 3 months ago

smlambert commented 5 months ago

Release Summary Report for release-openjdk21-pipeline

Warning: The release summary report is not yet complete. Currently, it only contains partial results.

Report generated at: Wed, 17 Apr 2024 12:19:30 GMT

TRSS Build and TRSS Grid View Jenkins Build URL https://ci.adoptium.net/job/build-scripts/job/release-openjdk21-pipeline/32/ Started by upstream project "build-scripts/utils/releaseTrigger_jdk21u" build number 2460 at 4/17/2024, 2:42:20 AM


✔️ results in these Tables means the activity has successfully completed. ⏳ results means that we are actively working on closing off the runs needed for this version, platform, binaryType. ⛔ means there is no build planned for that version/platform combination. ⏸️ means activity not yet started.

Platform Status Results Actions
x86-64_mac ✔️ comment
x86-64_linux ✔️ comment
x86-64_windows ✔️ comment
aarch64_linux ✔️ comment
aarch64_mac ✔️ comment
x86-64_alpine-linux ✔️ comment
ppc64le_linux ✔️ comment
s390x_linux ✔️ comment
ppc64_aix ✔️ all pass
aarch64_alpine-Linux ✔️ comment
riscv64_linux ✔️ comment

Rerun and grinder tap files Test_openjdk21_hs_extended.openjdk_x86-64_windows_rerun.tap.txt Grinder_20240418142004_JDK21Win64.tap.txt

smlambert commented 5 months ago

aarch64_mac

Test_openjdk21_hs_extended.openjdk_aarch64_mac ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_aarch64_mac_rerun ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode) ```

Test_openjdk21_hs_extended.openjdk_aarch64_mac_testList_0 ⚠️ UNSTABLE ⚠️ jdk_nio_0 => deep history 15/17 passed | possible issues jdk_net_0 => deep history 14/15 passed | possible issues

jdk_nio_0
output
Failed test cases: TEST: java/nio/channels/DatagramChannel/AdaptorMulticasting.java TEST: java/nio/channels/DatagramChannel/BasicMulticastTests.java Test results: passed: 461; failed: 2


[**Test_openjdk21_hs_extended.openjdk_aarch64_mac_testList_1**](https://ci.adoptium.net/job/Test_openjdk21_hs_extended.openjdk_aarch64_mac_testList_1/6/) ⚠️ UNSTABLE ⚠️
[jdk_jfr_1](https://trss.adoptium.net/output/test?id=661fb67a879917006e9b2513) => [deep history 5/15 passed](https://trss.adoptium.net/deepHistory?testId=661fb67a879917006e9b2513) | [possible issues](https://trss.adoptium.net/possibleIssues?buildId=661fb55c879917006e9b107b&buildName=Test_openjdk21_hs_extended.openjdk_aarch64_mac_testList_1&testId=661fb67a879917006e9b2513&testName=jdk_jfr_1)

Dumped recording, 329.4 kB written to:

/Users/admin/workspace/workspace/Test_openjdk21_hs_extended.openjdk_aarch64_mac_testList_1/aqa-tests/TKG/output_17133402063425/jdk_jfr_1/work/scratch/1/TestJCMdDump.jfr ]; stderr: [] exitValue = 0

java.lang.OutOfMemoryError: Java heap space at jdk.jfr.jcmd.TestJcmdDump.jfrDump(TestJcmdDump.java:107) at jdk.jfr.jcmd.TestJcmdDump.main(TestJcmdDump.java:87) at java.base/java.lang.invoke.LambdaForm$DMH/0x00000070000c1000.invokeStatic(LambdaForm$DMH) at java.base/java.lang.invoke.LambdaForm$MH/0x00000070000c2400.invoke(LambdaForm$MH) at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:154) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) at java.base/java.lang.Thread.runWith(Thread.java:1596) at java.base/java.lang.Thread.run(Thread.java:1583)

JavaTest Message: Test threw exception: java.lang.OutOfMemoryError: Java heap space JavaTest Message: shutting down test

STATUS:Failed.`main' threw exception: java.lang.OutOfMemoryError: Java heap space


- Rerun in [Grinder/9680](https://ci.adoptium.net/job/Grinder/9680) passes [Grinder_20240417133707_aarch64Mac_JDK21.tap.txt](https://github.com/adoptium/aqa-tests/files/15012279/Grinder_20240417133707_aarch64Mac_JDK21.tap.txt)
- Deep history shows this was passing consistently on Orka machines for some time, then starts failing consistently starting somewhere between Feb 4 and Feb 7, 2024, **ACTION**: check what changed in code, tests, or infra between that period of time
<img width="1254" alt="Screenshot 2024-04-17 at 9 30 05 AM" src="https://github.com/adoptium/aqa-tests/assets/2836948/9528c0ec-aab3-4b41-899a-93f49d7a4e2b">
smlambert commented 5 months ago

aarch64_alpine-Linux

Test_openjdk21_hs_extended.openjdk_aarch64_alpine-linux ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_aarch64_alpine-linux_rerun ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```
smlambert commented 5 months ago

aarch64_linux

Test_openjdk21_hs_extended.openjdk_aarch64_linux ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_aarch64_linux_rerun ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```

Test_openjdk21_hs_extended.openjdk_aarch64_linux_testList_0 ⚠️ UNSTABLE ⚠️ jdk_build_0 => deep history 18/21 passed | possible issues

Test_openjdk21_hs_extended.openjdk_aarch64_linux_testList_2 ⚠️ UNSTABLE ⚠️ jdk_jdi_0 => deep history 6/10 passed | possible issues jdk_build_1 => deep history 7/10 passed | possible issues

smlambert commented 5 months ago

ppc64le_linux

Test_openjdk21_hs_extended.openjdk_ppc64le_linux ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_ppc64le_linux_rerun ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```

Test_openjdk21_hs_extended.openjdk_ppc64le_linux_testList_0 ⚠️ UNSTABLE ⚠️ jvm_compiler_1 => deep history 14/22 passed | possible issues jdk_net_0 => deep history 10/16 passed | possible issues jdk_build_0 => deep history 18/22 passed | possible issues

Test_openjdk21_hs_extended.openjdk_ppc64le_linux_testList_1 ⚠️ UNSTABLE ⚠️ jvm_compiler_0 => deep history 8/15 passed | possible issues

Test_openjdk21_hs_extended.openjdk_ppc64le_linux_testList_2 ⚠️ UNSTABLE ⚠️ jdk_jdi_0 => deep history 11/16 passed | possible issues jdk_jdi_1 => deep history 19/23 passed | possible issues jdk_build_1 => deep history 12/16 passed | possible issues

smlambert commented 5 months ago

x86-64_windows

Grinder_20240418142004_JDK21Win64.tap.txt

Test_openjdk21_hs_extended.openjdk_x86-64_windows ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_x86-64_windows_rerun ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```

Test_openjdk21_hs_extended.openjdk_x86-64_windows_testList_0 ⚠️ UNSTABLE ⚠️ jdk_tools_0 => deep history 0/10 passed | possible issues jdk_tools_1 => deep history 0/11 passed | possible issues

Test_openjdk21_hs_extended.openjdk_x86-64_windows_testList_1 ⚠️ UNSTABLE ⚠️ jdk_security3_1 => deep history 8/11 passed | possible issues jdk_instrument_1 => deep history 16/17 passed | possible issues jdk_imageio_0 => deep history 16/17 passed | possible issues

smlambert commented 5 months ago

x86-64_linux

Test_openjdk21_hs_extended.openjdk_x86-64_linux ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_x86-64_linux_rerun ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```

Test_openjdk21_hs_extended.openjdk_x86-64_linux_testList_0 ⚠️ UNSTABLE ⚠️ jdk_build_0 => deep history 13/15 passed | possible issues - Grinder/9689

Test_openjdk21_hs_extended.openjdk_x86-64_linux_testList_1 ⚠️ UNSTABLE ⚠️ jdk_management_0 => deep history 9/10 passed | possible issues jdk_management_1 => deep history 9/10 passed | possible issues jdk_jfr_1 => deep history 5/10 passed | possible issues

Test_openjdk21_hs_extended.openjdk_x86-64_linux_testList_2 ⚠️ UNSTABLE ⚠️ jdk_build_1 => deep history 4/6 passed | possible issues

pmikova commented 5 months ago

x86-64_mac

Test_openjdk21_hs_extended.openjdk_x86-64_mac ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_x86-64_mac_rerun ✅ SUCCESS ✅ Rerun all

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode) ```

Test_openjdk21_hs_extended.openjdk_x86-64_mac_testList_1 ⚠️ UNSTABLE ⚠️ jdk_jfr_1 => deep history 6/9 passed | possible issues

The test failure was caused by a OOM:


[2024-04-17T23:22:37.698Z] /Users/jenkins/workspace/Test_openjdk21_hs_extended.openjdk_x86-64_mac_testList_1/aqa-tests/TKG/output_171338632273/jdk_jfr_1/work/scratch/1/TestJCMdDump.jfr
[2024-04-17T23:22:37.698Z] ];
[2024-04-17T23:22:37.698Z]  stderr: []
[2024-04-17T23:22:37.698Z]  exitValue = 0
[2024-04-17T23:22:37.698Z] 
[2024-04-17T23:22:37.698Z] java.lang.OutOfMemoryError: Java heap space
[2024-04-17T23:22:37.698Z]  at jdk.jfr.jcmd.TestJcmdDump.jfrDump(TestJcmdDump.java:107)
[2024-04-17T23:22:37.698Z]  at jdk.jfr.jcmd.TestJcmdDump.main(TestJcmdDump.java:87)
[2024-04-17T23:22:37.698Z]  at java.base/java.lang.invoke.LambdaForm$DMH/0x00000001480c1000.invokeStatic(LambdaForm$DMH)
[2024-04-17T23:22:37.698Z]  at java.base/java.lang.invoke.LambdaForm$MH/0x00000001480c2400.invoke(LambdaForm$MH)
[2024-04-17T23:22:37.698Z]  at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
[2024-04-17T23:22:37.698Z]  at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:154)
[2024-04-17T23:22:37.698Z]  at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[2024-04-17T23:22:37.698Z]  at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[2024-04-17T23:22:37.698Z]  at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
[2024-04-17T23:22:37.698Z]  at java.base/java.lang.Thread.runWith(Thread.java:1596)
[2024-04-17T23:22:37.698Z]  at java.base/java.lang.Thread.run(Thread.java:1583)
[2024-04-17T23:22:37.698Z] 
[2024-04-17T23:22:37.698Z] JavaTest Message: Test threw exception: java.lang.OutOfMemoryError: Java heap space
[2024-04-17T23:22:37.698Z] JavaTest Message: shutting down test

And was fixed by a re-run

The machine with the OOM was: test-macstadium-macos1014-x64-1 And the machine with the successful rerun was: test-macincloud-macos1201-x64-2

pmikova commented 5 months ago

s390x_linux

Test_openjdk21_hs_extended.openjdk_s390x_linux ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_s390x_linux_rerun ✅ SUCCESS ✅ Rerun all

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```

Test_openjdk21_hs_extended.openjdk_s390x_linux_testList_0 ⚠️ UNSTABLE ⚠️ jdk_net_0 => deep history 9/16 passed | possible issues jdk_nio_0 => deep history 10/16 passed | possible issues

Test_openjdk21_hs_extended.openjdk_s390x_linux_testList_1 ⚠️ UNSTABLE ⚠️ jdk_management_0 => deep history 8/16 passed | possible issues jdk_management_1 => deep history 8/15 passed | possible issues

The test issues were caused by a timeout:

jdk_management

[2024-04-17T17:45:49.361Z]  stderr: ['2024-04-17 17:45:25 FINE    Test started. 
[2024-04-17T17:45:49.361Z] ''2024-04-17 17:45:25 FINE    Listening for multicast packets at 224.0.23.178:7095 
[2024-04-17T17:45:49.361Z] ''2024-04-17 17:45:25 FINE    Waiting for 3 packets with jdp.name=91d22bef-bba4-49d0-88ee-4db3c9ac30f9 
[2024-04-17T17:45:49.361Z] ''2024-04-17 17:45:25 FINE    Pause in between packets is: 1 seconds. 
[2024-04-17T17:45:49.361Z] ''2024-04-17 17:45:25 FINE    Timeout set to 10 seconds. 
[2024-04-17T17:45:49.361Z] ''2024-04-17 17:45:35 SEVERE  Timed out waiting for JDP packet. Should arrive within 1 seconds, but waited for 10 seconds. 
[2024-04-17T17:45:49.361Z] ''2024-04-17 17:45:35 SEVERE  Timed out waiting for JDP packet. Should arrive within 1 seconds, but waited for 10 seconds. 
[2024-04-17T17:45:49.361Z] 'Exception in thread "main" java.lang.Exception: Timed out waiting for JDP packet. Should arrive within 1 seconds, but waited for 10 seconds.
[2024-04-17T17:45:49.362Z]  at JdpOnTestCase.onSocketTimeOut(JdpOnTestCase.java:75)
[2024-04-17T17:45:49.362Z]  at JdpTestCase.run(JdpTestCase.java:77)
[2024-04-17T17:45:49.362Z]  at JdpOnTestCase.main(JdpOnTestCase.java:105)
[2024-04-17T17:45:49.362Z] Caused by: java.net.SocketTimeoutException: Receive timed out
[2024-04-17T17:45:49.362Z]  at java.base/sun.nio.ch.DatagramChannelImpl.tryBlockingReceive(DatagramChannelImpl.java:760)
[2024-04-17T17:45:49.362Z]  at java.base/sun.nio.ch.DatagramChannelImpl.blockingReceive(DatagramChannelImpl.java:692)
[2024-04-17T17:45:49.362Z]  at java.base/sun.nio.ch.DatagramSocketAdaptor.receive(DatagramSocketAdaptor.java:204)
[2024-04-17T17:45:49.362Z]  at java.base/java.net.DatagramSocket.receive(DatagramSocket.java:714)
[2024-04-17T17:45:49.362Z]  at JdpTestCase.run(JdpTestCase.java:74)
[2024-04-17T17:45:49.362Z]  ... 1 more
[2024-04-17T17:45:49.362Z] ]
[2024-04-17T17:45:49.362Z]  exitValue = 1
[2024-04-17T17:45:49.362Z] 
[2024-04-17T17:45:49.362Z] java.lang.RuntimeException: Expected to get exit value of [0], exit value is: [1]
[2024-04-17T17:45:49.362Z]  at jdk.test.lib.process.OutputAnalyzer.shouldHaveExitValue(OutputAnalyzer.java:490)
[2024-04-17T17:45:49.362Z]  at DynamicLauncher.run(DynamicLauncher.java:61)
[2024-04-17T17:45:49.362Z]  at JdpDefaultsTest.main(JdpDefaultsTest.java:46)
[2024-04-17T17:45:49.362Z]  at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[2024-04-17T17:45:49.362Z]  at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[2024-04-17T17:45:49.362Z]  at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
[2024-04-17T17:45:49.362Z]  at java.base/java.lang.Thread.run(Thread.java:1583)
[2024-04-17T17:45:49.362Z] 
[2024-04-17T17:45:49.362Z] JavaTest Message: Test threw exception: java.lang.RuntimeException: Expected to get exit value of [0], exit value is: [1]
[2024-04-17T17:45:49.362Z] JavaTest Message: shutting down test`

jdk_nio

ACTION: main -- Error. Program `/home/jenkins/workspace/Test_openjdk21_hs_extended.openjdk_s390x_linux_testList_0/jdkbinary/j2sdk-image/bin/java' timed out (timeout set to 960000ms, elapsed time including timeout handling was 960788ms).

jdk_net

TEST RESULT: Error. Program `/home/jenkins/workspace/Test_openjdk21_hs_extended.openjdk_s390x_linux_testList_0/jdkbinary/j2sdk-image/bin/java' timed out (timeout set to 960000ms, elapsed time including timeout handling was 960553ms).

The timeouted tests ran on the following machines: test-marist-ubuntu2204-s390x-1 test-marist-ubuntu2004-s390x-1

And it was fixed by a re-run on: test-docker-sles12-s390x-1

smlambert commented 5 months ago

x86-64_alpine-linux

java/nio/file/DirectoryStream/SecureDS.java.SecureDS build/AbsPathsInImage.java.AbsPathsInImage

Test_openjdk21_hs_extended.openjdk_x86-64_alpine-linux ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_x86-64_alpine-linux_rerun ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```

Test_openjdk21_hs_extended.openjdk_x86-64_alpine-linux_testList_0 ⚠️ UNSTABLE ⚠️ jdk_nio_0 => deep history 4/10 passed | possible issues

Test_openjdk21_hs_extended.openjdk_x86-64_alpine-linux_testList_2 ⚠️ UNSTABLE ⚠️ jdk_nio_1 => deep history 10/17 passed | possible issues jdk_build_0 => deep history 0/10 passed | possible issues jdk_build_1 => deep history 1/17 passed | possible issues

smlambert commented 4 months ago

riscv64_linux

Test_openjdk21_hs_extended.openjdk_riscv64_linux ⚠️ UNSTABLE ⚠️

Test_openjdk21_hs_extended.openjdk_riscv64_linux_rerun ✅ SUCCESS ✅ Rerun all

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```

Test_openjdk21_hs_extended.openjdk_riscv64_linux_testList_2 ⚠️ UNSTABLE ⚠️ jdk_jmx_0 => deep history 1/2 passed | possible issues

Test_openjdk21_hs_sanity.openjdk_riscv64_linux ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```

jdk_lang_0 => deep history 3/7 passed | possible issues

Test_openjdk21_hs_sanity.openjdk_riscv64_linux_rerun ⚠️ UNSTABLE ⚠️ Rerun failed

java -version ``` openjdk version "21.0.3" 2024-04-16 LTS OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing) ```