search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.27k stars 721 forks source link

JEP 393: Foreign-Memory Access API (Third Incubator) #11027

Closed pshipton closed 2 years ago

pshipton commented 3 years ago

See https://openjdk.java.net/jeps/393

pshipton commented 3 years ago

@tajila

tajila commented 3 years ago

This has been assigned to @babsingh

babsingh commented 3 years ago

https://github.com/eclipse/openj9/pull/11182 introduces the following two native stubs:

  1. Java_jdk_internal_misc_ScopedMemoryAccess_registerNatives: stays empty since our tooling handles the registration of Java native methods.
  2. Java_jdk_internal_misc_ScopedMemoryAccess_closeScope0: currently, throws an assertion; needs an implementation.

New tests and failures are seen for JEP393 (Foreign Memory Access API's Third Incubator):

[SUMMARY] Test results: passed: 20; failed: 4; error: 1

Failed: java/foreign/TestSharedAccess.java
Failed: java/foreign/TestSpliterator.java
Failed: java/foreign/TestByteBuffer.java
Failed: java/foreign/TestHandshake.java
Failed: java/foreign/TestCleaner.java

More details about the failures can be found in the following log: jep393_tests_19Nov20.log.

Need to implement Java_jdk_internal_misc_ScopedMemoryAccess_closeScope0 in order to resolve some of the above JEP393 failures.

babsingh commented 3 years ago

More JEP393 failures: https://github.com/eclipse/openj9/issues/11135#issuecomment-724814903.

JasonFengJ9 commented 3 years ago

Note: following tests are excluded via https://github.com/AdoptOpenJDK/openjdk-tests/pull/2085

Failed: java/foreign/TestSharedAccess.java
Failed: java/foreign/TestSpliterator.java
Failed: java/foreign/TestByteBuffer.java
Failed: java/foreign/TestHandshake.java
Failed: java/foreign/TestCleaner.java
JasonFengJ9 commented 3 years ago

More excludes for java/foreign tests via https://github.com/AdoptOpenJDK/openjdk-tests/pull/2088 before the implementation is ready.

java/foreign/stackwalk/TestStackWalk.java 
java/foreign/valist/VaListTest.java 
java/foreign/StdLibTest.java 
java/foreign/TestAdaptVarHandles.java 
java/foreign/TestAddressHandle.java 
java/foreign/TestDowncall.java 
java/foreign/TestIllegalLink.java 
java/foreign/TestIntrinsics.java 
java/foreign/TestMemoryAccess.java 
java/foreign/TestMemoryAccessStatics.java 
java/foreign/TestMemoryAlignment.java 
java/foreign/TestMemoryCopy.java 
java/foreign/TestMemoryHandleAsUnsigned.java 
java/foreign/TestMismatch.java 
java/foreign/TestNativeScope.java 
java/foreign/TestNulls.java 
java/foreign/TestRebase.java 
java/foreign/TestSegments.java 
java/foreign/TestSlices.java 
java/foreign/TestTypeAccess.java 
java/foreign/TestUpcall.java 
java/foreign/TestUpcallHighArity.java 
java/foreign/TestUpcallStubs.java 
java/foreign/TestVarArgs.java 
java/foreign/TestVarHandleCombinators.java

fyi @tajila @babsingh @pshipton

babsingh commented 3 years ago

More excludes for java/foreign tests via AdoptOpenJDK/openjdk-tests#2088

Do all the tests, which are being excluded, fail? As per the log file in https://github.com/eclipse/openj9/issues/11027#issuecomment-730849265 (generated 2 weeks ago), few of these tests pass.

tajila commented 3 years ago

More excludes for java/foreign tests

Some of these tests are related to JEP389 rather than JEP393

babsingh commented 3 years ago

Some of these tests are related to JEP389 rather than JEP393

JEP389 tests also run as part of the jdk_foreign test suite. I do not think it's possible to run JEP389 and JEP393 tests separately. So, I have identified the JEP389 tests below:

java/foreign/stackwalk/TestStackWalk.java 
java/foreign/valist/VaListTest.java 
java/foreign/StdLibTest.java 
java/foreign/TestDowncall.java 
java/foreign/TestIllegalLink.java 
java/foreign/TestIntrinsics.java 
java/foreign/TestUpcall.java 
java/foreign/TestUpcallHighArity.java 
java/foreign/TestUpcallStubs.java 
java/foreign/TestVarArgs.java
JasonFengJ9 commented 3 years ago

Do all the tests, which are being excluded, fail? As per the log file in #11027 (comment) (generated 2 weeks ago), few of these tests pass.

Yes, all excluded tests still fail in internal tests.

JasonFengJ9 commented 3 years ago

Some of these tests are related to JEP389 rather than JEP393

Will check and refer JEP 389 instead.

tajila commented 3 years ago

We need to track them differently as there is a chance that JEP393 will be ready for 0.25 and JEP389 will not

JasonFengJ9 commented 3 years ago

From an internal build Test_openjdknext_j9_sanity.openjdk_x86-64_linux_Nightly/363:

22:36:40  --------------------------------------------------
22:36:43  TEST: java/foreign/TestAdaptVarHandles.java

22:36:43  Caused by: java.lang.ClassCastException: java.lang.invoke.DirectHandle incompatible with java.lang.invoke.BoundMethodHandle
22:36:43    at java.base/java.lang.invoke.VarHandles.noCheckedExceptions(VarHandles.java:636)
22:36:43    at java.base/java.lang.invoke.VarHandles.filterCoordinates(VarHandles.java:476)
22:36:43    at java.base/java.lang.invoke.MethodHandleImpl$1.filterCoordinates(MethodHandleImpl.java:1790)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryHandles.filterCoordinates(MemoryHandles.java:374)
22:36:43    at jdk.incubator.foreign/jdk.internal.foreign.Utils.fixUpVarHandle(Utils.java:92)
22:36:43    at jdk.incubator.foreign/jdk.internal.foreign.LayoutPath.dereferenceHandle(LayoutPath.java:159)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.lambda$varHandle$3(MemoryLayout.java:411)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout$$Lambda$55/0x0000000000000000.apply(Unknown Source)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.computePathOp(MemoryLayout.java:457)
22:36:43    at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.varHandle(MemoryLayout.java:411)
22:36:43    at TestAdaptVarHandles.<clinit>(TestAdaptVarHandles.java:90)
22:36:43    ... 30 more
22:36:43  

22:36:43  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestAdaptVarHandles: null
Similar exception as above
22:36:43  ===============================================
22:36:43  java/foreign/TestAddressHandle.java
22:36:43  Total tests run: 6, Failures: 4, Skips: 2
22:36:43  ===============================================
Similar exception as above
22:37:09  ===============================================
22:37:09  java/foreign/TestMemoryAccess.java
22:37:09  Total tests run: 256, Failures: 252, Skips: 0
22:37:09  ===============================================
Similar exception as above
22:37:09  ===============================================
22:37:09  java/foreign/TestMemoryAccessStatics.java
22:37:09  Total tests run: 50, Failures: 50, Skips: 0
22:37:09  ===============================================
Similar exception as above
22:37:13  ===============================================
22:37:13  java/foreign/TestMemoryAlignment.java
22:37:13  Total tests run: 117, Failures: 63, Skips: 0
22:37:13  ===============================================
Similar exception as above
22:37:13  --------------------------------------------------
22:37:13  TEST: java/foreign/TestMemoryCopy.java
22:37:13  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestMemoryCopy: null
Similar exception as above
22:37:13  ===============================================
22:37:13  java/foreign/TestMemoryHandleAsUnsigned.java
22:37:13  Total tests run: 2360, Failures: 2360, Skips: 0
22:37:13  ===============================================
22:37:16  --------------------------------------------------
22:37:16  TEST: java/foreign/TestNulls.java
Similar exception as above
22:37:16  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestNulls: null
22:37:16  --------------------------------------------------
22:37:20  TEST: java/foreign/TestRebase.java
Similar exception as above
22:37:20  TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 1
Similar exception as above
22:37:20  ===============================================
22:37:20  java/foreign/TestSegments.java
22:37:20  Total tests run: 120, Failures: 14, Skips: 1
22:37:20  ===============================================
22:37:20  --------------------------------------------------
22:37:23  TEST: java/foreign/TestSlices.java
Similar exception as above
22:37:23  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestSlices: null
22:37:23  --------------------------------------------------
22:37:23  TEST: java/foreign/TestTypeAccess.java
Similar exception as above
22:37:23  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestTypeAccess: null
Similar exception as above
22:37:26  ===============================================
22:37:26  java/foreign/TestVarHandleCombinators.java
22:37:26  Total tests run: 13, Failures: 6, Skips: 0
22:37:26  ===============================================
22:37:13  --------------------------------------------------
22:37:13  TEST: java/foreign/TestMismatch.java
Similar exception as above
22:37:13  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestMismatch: null
22:37:13  --------------------------------------------------
22:37:16  TEST: java/foreign/TestNativeScope.java

Similar exception as above and ASSERTION failure:

22:37:16  STDERR:
22:37:16  WARNING: Using incubator modules: jdk.incubator.foreign
22:37:16  03:37:13.015 0x1c6800   j9jcl.347    *   ** ASSERTION FAILED ** at common/jdk_internal_misc_ScopedMemoryAccess.cpp:40: ((0 ))
22:37:16  JVMDUMP039I Processing dump event "traceassert", detail "" at 2020/12/02 19:37:13 - please wait.
babsingh commented 3 years ago

Re: https://github.com/eclipse/openj9/issues/11027#issuecomment-738965440 and https://github.com/eclipse/openj9/issues/11027#issuecomment-738929992

Locally, some of the jdk/foreign tests pass for me because I have the following fix applied to my JDK16 extensions repo: https://github.com/ibmruntimes/openj9-openjdk-jdk/pull/248. This fix adds support for OpenJ9 VarHandles but at the same time, it breaks support for OpenJDK VarHandles. This fix will not be merged since we plan to use OpenJDK VarHandles in JDK16. Related: https://github.com/eclipse/openj9/issues/11135#issuecomment-724814903. For the nightly openjdk sanity builds, @JasonFengJ9's failures are correctly documented.

babsingh commented 3 years ago

Below is the updated list of JEP393 test failures after https://github.com/ibmruntimes/openj9-openjdk-jdk16/pull/9 is merged for the 0.25 release:

java/foreign/TestByteBuffer.java
java/foreign/TestSharedAccess.java
java/foreign/TestSpliterator.java
java/foreign/TestHandshake.java
java/foreign/TestCleaner.java
java/foreign/TestNativeScope.java

The above tests fail because of 

** ASSERTION FAILED ** at common/jdk_internal_misc_ScopedMemoryAccess.cpp:40: ((0 ))

The above assertion failure is being tracked via this issue https://github.com/eclipse/openj9/issues/11027#issuecomment-730849265 and https://github.com/eclipse/openj9/issues/11360 (duplicate).

babsingh commented 3 years ago

JEP393 is targeted for JDK17 (0.28 release).

pshipton commented 1 year ago

Un-exclude some of the tests excluded under this issue via https://github.com/adoptium/aqa-tests/pull/4211

The remaining jdk17 excludes done via this issue are still excluded on jdk19 under the following (open) issues: https://github.com/eclipse-openj9/openj9/issues/13994 java/foreign/valist/VaListTest.java java/foreign/TestDowncall.java java/foreign/TestIntrinsics.java java/foreign/TestVarArgs.java

https://github.com/eclipse-openj9/openj9/issues/13999 java/foreign/StdLibTest.java java/foreign/TestUpcall.java java/foreign/TestUpcallHighArity.java

https://github.com/eclipse-openj9/openj9/issues/14002 - perm exclude java/foreign/TestIllegalLink.java

https://github.com/eclipse-openj9/openj9/issues/14134 java/foreign/TestMemoryAccess.java java/foreign/TestNulls.java

plus https://github.com/eclipse-openj9/openj9/issues/13993 which is a permanent exclude java/foreign/stackwalk/TestStackWalk.java#default_gc java/foreign/stackwalk/TestStackWalk.java#shenandoah java/foreign/stackwalk/TestStackWalk.java#zgc