search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 720 forks source link

Annotation testing failures #1765

Open jdekonin opened 6 years ago

jdekonin commented 6 years ago

Running "_sanity" testing on Windows 2012 is now failing. The sha tested against is a31cbae7, the last good sha was : 66fc23ae

Diff : https://github.com/eclipse/openj9/compare/66fc23ae...a31cbae7

01:20:36 =========================================== ... (more...) 01:20:36 =============================================== 01:20:36 GeneralTest suite 01:20:36 Total tests run: 644, Failures: 27, Skips: 0 01:20:36 ===============================================

Tests : org.openj9.test.annotation.Test_Annotation.* org.openj9.test.annotationPackage.Test_PackageAnnotation.test_elementFormDefault org.openj9.test.annotationPackage.Test_PackageAnnotation.test_elementFormDefault org.openj9.test.annotationPackage.Test_PackageAnnotation2.test_isAnnotationPresent org.openj9.test.annotationPackage.Test_PackageAnnotation2.test_isAnnotationPresent

pshipton commented 6 years ago

Is the failure on Java 8 or Java 9?

pshipton commented 6 years ago

Joe said the failure only occurs on Java 9. It happens on the Jenkins build but not the Axxon build (first occurred in 385259, but a number of previous builds didn't run on win2012 machines). The OpenJ9 changes were the only changes.

pshipton commented 6 years ago

The failure appears intermittent, or machine specific as it seems to only be failing on Windows 2012. Failed on multiple machines win2012x64rt2 and win2012x64rt3. First failure was Apr 24, 2018 5:01 PM. Last time it passed on win2012 was Apr 24, 2018 12:43 PM, but I don't know how to trace that to the build which started the test. The Jenkins build history doesn't go back far enough.

pshipton commented 6 years ago 
01:20:36         FAILED: test_package_Annotation_annotation
01:20:36         org.openj9.test.annotation.AnnotationNotFoundException: ValueAnnotation not found on package org.openj9.test.annotation
01:20:36            at org.openj9.test.annotation.Test_Annotation.test_package_Annotation_annotation(Test_Annotation.java:253) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/functional/Java8andUp/GeneralTest.jar)
01:20:36            at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9.0.4-internal/Native Method) from jrt:/java.base
01:20:36            at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9.0.4-internal/NativeMethodAccessorImpl.java:62) from jrt:/java.base
01:20:36            at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9.0.4-internal/DelegatingMethodAccessorImpl.java:43) from jrt:/java.base
01:20:36            at java.lang.reflect.Method.invoke(java.base@9.0.4-internal/Method.java:564) from jrt:/java.base
01:20:36            at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.internal.Invoker.invokeMethod(Invoker.java:580) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:716) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:988) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.TestRunner.privateRun(TestRunner.java:648) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.TestRunner.run(TestRunner.java:505) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.SuiteRunner.runTest(SuiteRunner.java:455) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.SuiteRunner.run(SuiteRunner.java:364) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.TestNG.runSuitesLocally(TestNG.java:1137) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.TestNG.runSuites(TestNG.java:1049) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.TestNG.run(TestNG.java:1017) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.TestNG.privateMain(TestNG.java:1354) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
01:20:36            at org.testng.TestNG.main(TestNG.java:1323) from jdk.internal.loader.ClassLoaders$AppClassLoader@a9fd01a2(file:/C:/Users/j9build/jenkins/workspace/Test-Sanity-JDK9-win_x86-64_cmprssptrs/openj9/jvmtest/TestConfig/lib/testng.jar)
jdekonin commented 6 years ago

Occurs on JDK8 builds as well. Hit the same 2012 machine, win2012x64rt3.

vsebe commented 6 years ago

Test passes on a Windows 7(win7x64rt2) machine on JDK8.

DanHeidinga commented 6 years ago

The test is looking up annotations from the package: https://github.com/eclipse/openj9/blob/d6a3535ce28ffaa0602c43bd745635ffc2bc7858/test/functional/Java8andUp/src/org/openj9/test/annotation/Test_Annotation.java#L250-L255

DanHeidinga commented 6 years ago

Given the intermittent nature of this, we should look at how to modify the test to catch the failure and validate if this is a "real" failure or a machine (network? filesystem?) issue

jdekonin commented 6 years ago

I have this job failing on an internal win2008 and AIX7.1 systems. And I can't find the job at the moment but I am 99% sure I've seen this fail on x86 CentOS6.9

ValueAnnotation.class can be found in generaltest.jar. So far I haven't been able to catch a failing job and check the jar to see if its actually there. I have also compiled and checked for the classes existence in a 20x loop with no apparent problems.

jdekonin commented 6 years ago

I have a grinder job like what is setup at AdoptOpenJDK and currently have a 10x on the same machine running. Next I'll look at adding more debugging around the test itself.