search

adoptium / aqa-tests

Home of test infrastructure for Adoptium builds
https://adoptium.net/aqavit
Apache License 2.0
128 stars 308 forks source link

Evaluate jdk_math regression and exclude sparcv9 solaris jdk8 problem tests #4150

Open Haroon-Khel opened 1 year ago

Haroon-Khel commented 1 year ago

The following tests fail on the current (jdk8u352-b08) and last (8u345b01) release of jdk 8 on sparcv9 solaris and should be excluded

jdk_jfr

jdk/jfr/javaagent/TestLoadedAgent.java jdk/jfr/jcmd/TestJcmdSaveToFile.java jdk/jfr/jcmd/TestJcmdStartWithSettings.java jdk/jfr/jcmd/TestJcmdStopInvalidFile.java jdk/jfr/jcmd/TestJcmdStopReadOnlyFile.java

jdk_security3

javax/net/ssl/Stapling/HttpsUrlConnClient.java.HttpsUrlConnClient javax/net/ssl/Stapling/SSLSocketWithStapling.java.SSLSocketWithStapling javax/net/ssl/TLSCommon/ConcurrentClientAccessTest.java.ConcurrentClientAccessTest javax/net/ssl/TLSCommon/TestSessionLocalPrincipal.java.TestSessionLocalPrincipal sun/security/pkcs11/KeyStore/ClientAuth.sh.ClientAuth sun/security/pkcs11/fips/ClientJSSEServerJSSE.java.ClientJSSEServerJSSE

System

OAuthTest_0

jdk_math

java/math/BigInteger/LargeValueExceptions.java

hotspot_jre

compiler/8209951/TestCipherBlockChainingEncrypt.java

smlambert commented 1 year ago

As noted in Slack, jdk_math appears to be a regression (having passed in the July release seen in TRSS deep history, seen passing as recently as Sept 13 in https://ci.adoptopenjdk.net/job/Test_openjdk8_hs_sanity.openjdk_sparcv9_solaris/180/, but failing in October release).

We can rerun the test target using the SHA from Sept 13th (see contents of testenv.properties from that run below), to rule out that its related to test changes such as 8282252: Improve BigInteger/Decimal validation. (see Grinder/6285)

Sept 13 testenv.properties (from a passing run)

ADOPTOPENJDK_REPO=https://github.com/adoptium/aqa-tests.git
ADOPTOPENJDK_BRANCH=232c608b4a340bc512c69012ffbc7edb888832aa
TKG_REPO=https://github.com/adoptium/TKG.git
TKG_BRANCH=f3d79fd4e8a9b3eda5e4f1fdcce0439001b3c99e
OPENJ9_REPO=https://github.com/eclipse-openj9/openj9.git
OPENJ9_SHA=df2f3a3524957ff9e59c6861992f8195beb24aa6
JDK8_REPO=https://github.com/adoptium/jdk8u
JDK8_BRANCH=bb4f7da37aeb60ba494e4fc9e7d81d7df74bd52e

Running from present day test material (jdk8u-dev) against a Sept 5th EA build in Grinder 6286

smlambert commented 1 year ago

From Jan 2023 CPU, see https://github.com/adoptium/aqa-tests/issues/4258#issuecomment-1398963195 for the list of failing tests (copied below for convenience):

jdk_security1 (known and non-blocking, https://github.com/adoptium/aqa-tests/issues/4244):
TEST: java/security/cert/CertPathBuilder/targetConstraints/BuildEEBasicConstraints.java
TEST: java/security/cert/pkix/policyChanges/TestPolicy.java

jdk_security3:
TEST: javax/net/ssl/Stapling/HttpsUrlConnClient.java
TEST: javax/net/ssl/Stapling/SSLSocketWithStapling.java
TEST: javax/net/ssl/TLSCommon/ConcurrentClientAccessTest.java
TEST: javax/net/ssl/TLSCommon/TestSessionLocalPrincipal.java
TEST: sun/security/pkcs11/fips/ClientJSSEServerJSSE.java
TEST: sun/security/pkcs11/KeyStore/ClientAuth.sh
Test results: passed: 610; failed: 3; error: 3

hotspot_jre:
TEST: compiler/8209951/TestCipherBlockChainingEncrypt.java

jdk_jfr (pass on :
TEST: jdk/jfr/event/gc/stacktrace/TestG1OldAllocationPendingStackTrace.java
TEST: jdk/jfr/javaagent/TestLoadedAgent.java
TEST: jdk/jfr/jcmd/TestJcmdSaveToFile.java
TEST: jdk/jfr/jcmd/TestJcmdStartWithOptions.java
TEST: jdk/jfr/jcmd/TestJcmdStartWithSettings.java
TEST: jdk/jfr/jcmd/TestJcmdStopInvalidFile.java
TEST: jdk/jfr/jcmd/TestJcmdStopReadOnlyFile.java
Test results: passed: 411; failed: 6; error: 1

OAuthTest_0

jdk_math:
TEST: java/math/BigInteger/LargeValueExceptions.java
Test results: passed: 38; error: 1

also related: https://github.com/adoptium/aqa-tests/issues/3254