search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 721 forks source link

JTReg Test Failure: java/lang/StackWalker/DumpStackTest.java #6680

Closed M-Davies closed 2 years ago

M-Davies commented 5 years ago

Failure link

https://github.com/ibmruntimes/openj9-openjdk-jdk11/blob/openj9/test/jdk/java/lang/StackWalker/DumpStackTest.java

Consistent failure with jdk11-openj9 (all generic operating systems). Test passes on hotspot for all platforms

Java Version (when running locally)

openjdk 11.0.4 2019-07-16
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.4+11)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.15.1, JRE 11 Mac OS X amd64-64-Bit Compressed References 20190717_289 (JIT enabled, AOT enabled)
OpenJ9   - 0f66c6431
OMR      - ec782f26
JCL      - fa49279450 based on jdk-11.0.4+11)

Summary

The issue is a failed assertEquals() caused by an unexpected call frame in the consumeLambda() test.

Failure output

13:29:44  --- Actual ---
13:29:44  java.base/java.lang.Thread.getStackTrace(Thread.java:1169)
13:29:44  DumpStackTest.consumeLambda(DumpStackTest.java:147)
13:29:44  DumpStackTest.lambda$testLambda$0(DumpStackTest.java:129)
13:29:44  DumpStackTest$$Lambda$3.000000009FE08CD0.accept(Unknown Source)
13:29:44  DumpStackTest.testLambda(DumpStackTest.java:130)
13:29:44  DumpStackTest.main(DumpStackTest.java:45)
13:29:44  java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
13:29:44  java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
13:29:44  java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
13:29:44  java.base/java.lang.reflect.Method.invoke(Method.java:566)
13:29:44  com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
13:29:44  java.base/java.lang.Thread.run(Thread.java:831)
13:29:44  --- Expected ---
13:29:44  java.lang.Thread.getStackTrace(Thread.java)
13:29:44  DumpStackTest.consumeLambda(DumpStackTest.java)
13:29:44  DumpStackTest.lambda$testLambda$0(DumpStackTest.java)
13:29:44  DumpStackTest.testLambda(DumpStackTest.java)
13:29:44  DumpStackTest.main(DumpStackTest.java)
13:29:44  jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java)
13:29:44  jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java)
13:29:44  jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java)
13:29:44  java.lang.reflect.Method.invoke(Method.java)
13:29:44  java.lang.Thread.run(Thread.java)
13:29:44  STDERR:
13:29:44  java.lang.RuntimeException: StackTraceElements mismatch at index 3. Expected [DumpStackTest.testLambda(DumpStackTest.java)], but get [DumpStackTest$$Lambda$3.000000009FE08CD0.accept(Unknown Source)]
13:29:44    at DumpStackTest.assertEquals(DumpStackTest.java:227)
13:29:44    at DumpStackTest.assertStackTrace(DumpStackTest.java:219)
13:29:44    at DumpStackTest.consumeLambda(DumpStackTest.java:147)
13:29:44    at DumpStackTest.lambda$testLambda$0(DumpStackTest.java:129)
13:29:44    at DumpStackTest$$Lambda$3.000000009FE08CD0.accept(Unknown Source)
13:29:44    at DumpStackTest.testLambda(DumpStackTest.java:130)
13:29:44    at DumpStackTest.main(DumpStackTest.java:45)
13:29:44    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
13:29:44    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
13:29:44    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
13:29:44    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
13:29:44    at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
13:29:44    at java.base/java.lang.Thread.run(Thread.java:831)
13:29:44  
13:29:44  JavaTest Message: Test threw exception: java.lang.RuntimeException: StackTraceElements mismatch at index 3. Expected [DumpStackTest.testLambda(DumpStackTest.java)], but get [DumpStackTest$$Lambda$3.000000009FE08CD0.accept(Unknown Source)]
13:29:44  JavaTest Message: shutting down test
pdbain-ibm commented 5 years ago

This test is relying on unspecified details of the lambda implementation. I believe the OpenJ9 stack frame is correct, unless DumpStackTest$$Lambda$3.000000009FE08CD0.accept should be a hidden frame. @DanHeidinga do you concur?

M-Davies commented 5 years ago

openjdk_test_output (3).tar.gz Log file

M-Davies commented 5 years ago

Is it ok for me to close this @DanHeidinga if you're in agreement with Peter's comment?

DanHeidinga commented 5 years ago

What criteria is OpenJDK using to determine if the frame should be visible or not?

If it's that the class is loaded by Unsafe.defineAnonymousClass then OpenJ9 should mark those frames as invisible as well.

DanHeidinga commented 5 years ago

I think more investigation is needed here

M-Davies commented 5 years ago

I dig some digging using -Xdump: Unsafe loads MethodHandle but not the .accept frame. Neither Lambda or DumpStackTest are loaded using Unsafe

DanHeidinga commented 5 years ago

There may be a fix for this actually. It would be worthwhile building with this PR https://github.com/eclipse/openj9/pull/3627 and then retesting.

M-Davies commented 5 years ago

That seems to have fixed it. There is now a NullPointerError exception later in the code. The prefix that is attached is statically typed so it's possible the class name has changed in that time and the prefix is now incorrect. Will try running with contains() instead to see if that helps the test find the class name.

Error Output:

11:28:48  --- Actual ---
11:28:48  java.base/java.lang.Thread.getStackTrace(Thread.java:1169)
11:28:48  DumpStackTest.consumeLambda(DumpStackTest.java:147)
11:28:48  DumpStackTest.lambda$testLambda$0(DumpStackTest.java:129)
11:28:48  DumpStackTest.testLambda(DumpStackTest.java:130)
11:28:48  DumpStackTest.main(DumpStackTest.java:45)
11:28:48  java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
11:28:48  java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
11:28:48  java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
11:28:48  java.base/java.lang.reflect.Method.invoke(Method.java:566)
11:28:48  com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
11:28:48  java.base/java.lang.Thread.run(Thread.java:831)
11:28:48  null
11:28:48  --- Expected ---
11:28:48  java.lang.Thread.getStackTrace(Thread.java)
11:28:48  DumpStackTest.consumeLambda(DumpStackTest.java)
11:28:48  DumpStackTest.lambda$testLambda$0(DumpStackTest.java)
11:28:48  DumpStackTest.testLambda(DumpStackTest.java)
11:28:48  DumpStackTest.main(DumpStackTest.java)
11:28:48  jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java)
11:28:48  jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java)
11:28:48  jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java)
11:28:48  java.lang.reflect.Method.invoke(Method.java)
11:28:48  java.lang.Thread.run(Thread.java)
11:28:48  STDERR:
11:28:48  java.lang.NullPointerException
11:28:48    at DumpStackTest.assertStackTrace(DumpStackTest.java:217)
11:28:48    at DumpStackTest.consumeLambda(DumpStackTest.java:147)
11:28:48    at DumpStackTest.lambda$testLambda$0(DumpStackTest.java:129)
11:28:48    at DumpStackTest.testLambda(DumpStackTest.java:130)
11:28:48    at DumpStackTest.main(DumpStackTest.java:45)
11:28:48    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
11:28:48    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
11:28:48    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
11:28:48    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
11:28:48    at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
11:28:48    at java.base/java.lang.Thread.run(Thread.java:831)
11:28:48    at null
11:28:48  
11:28:48  JavaTest Message: Test threw exception: java.lang.NullPointerException
11:28:48  JavaTest Message: shutting down test
M-Davies commented 4 years ago

@DanHeidinga FYI, After working with the PR a bit today and modifying the test a little, inserting a try-catch for the NullPointerException at the place where it occurred, passes so this is the only issue with the test.

NEW STACK TRACE!! --- Actual --- DumpStackTest.getStackTrace(DumpStackTest.java:101) DumpStackTest.consumeLambda(DumpStackTest.java:148) DumpStackTest.lambda$testLambda$0(DumpStackTest.java:129) DumpStackTest.testLambda(DumpStackTest.java:130) DumpStackTest.main(DumpStackTest.java:45) java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.base/java.lang.reflect.Method.invoke(Method.java:566) com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) java.base/java.lang.Thread.run(Thread.java:834) null --- Expected --- DumpStackTest.getStackTrace(DumpStackTest.java) DumpStackTest.consumeLambda(DumpStackTest.java) DumpStackTest.lambda$testLambda$0(DumpStackTest.java) DumpStackTest.testLambda(DumpStackTest.java) DumpStackTest.main(DumpStackTest.java) jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java) jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java) jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) java.lang.reflect.Method.invoke(Method.java) java.lang.Thread.run(Thread.java) NullPointerThrown! <- (Same issue as before)


- The J9 and Hotspot implementations of `Thread.java` appear identical so I believe this is just an output bug.  
- The two streams (`Actual` and `Expected`) are also identical, other than the `null` frames, so the `NullPointerException` is caused by the stream's running out of objects to print
- From https://docs.oracle.com/javase/8/docs/api/java/util/Arrays.html and https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html

Throws: NullPointerException - if the result of the reduction is null


- A solution could be to add the try-catch in but I'm not sure if this would be accepted by the OpenJDK community (@smlambert Could something like this work? https://github.com/M-Davies/openj9-openjdk-jdk11/commit/2fa9485dbbe484a24aee8c8f347a2006857ee63f) 
- Or, alternatively, figure out what this null frame is and go from there.
fengxue-IS commented 2 years ago

Update: The NullPointer exception I believe came from the frames which was hidden by previous impl for #3627, as it filters the frame that is written into the result array but have no control over the array size during creation leaving null elements trailing. The new impl for #3627 that have been merged avoids this issue by running the filter during swalk call which updates stackframe count.

in local testing, this test passed for JDK18 and JDKnext, will try with JDK11 & JDK17 to confirm if this is fixed by #3627 for all versions and re-enable the test

fengxue-IS commented 2 years ago

local testing passed for JDK 11 & JDK 17, opened PR to re-enable testing