search

puniverse / quasar

Fibers, Channels and Actors for the JVM
http://docs.paralleluniverse.co/quasar/
Other
4.56k stars 575 forks source link

Illegal reflective access #321

Open ipkpjersi opened 5 years ago

ipkpjersi commented 5 years ago

Hello,

In newer versions of Java beyond 11, illegal reflective access will be denied by default and eventually it will no longer be allowed at all. Currently, Quasar throws an illegal reflective access warning as follows:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access using Lookup on co.paralleluniverse.concurrent.util.ThreadAccess (file:/home/user/Desktop/project/lib/quasar-core-0.8.0.jar) to class java.lang.Thread
WARNING: Please consider reporting this to the maintainers of co.paralleluniverse.concurrent.util.ThreadAccess
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Is there anything that can be done to fix this? I know that for example, netty has fixed their illegal reflective access: https://github.com/netty/netty/issues/8318 and pgjdbc has fixed their illegal reflective access as well: https://github.com/bolcom/pgjdbc/commit/586525744aa4c7973375f862a54043ea383d230d

I think fixing this is important to ensure that Quasar will continue working in Java versions beyond 11. I did not see any other issues mentioning this so I decided to open my own issue for this.

Thank you.

remmeier commented 5 years ago

It seems that there are also issues with OpenJ9 in the same area (in this case I tested with the Java 8 flavor in the context of R3 Corda):

Caused by: java.lang.AssertionError: java.lang.NoSuchFieldException: target
    at co.paralleluniverse.concurrent.util.ThreadAccess.<clinit>(ThreadAccess.java:80)
    at co.paralleluniverse.fibers.Fiber.<init>(Fiber.java:199)
    at co.paralleluniverse.fibers.Fiber.<init>(Fiber.java:457)
    at net.corda.node.services.statemachine.FlowStateMachineImpl.<init>(FlowStateMachineImpl.kt:48)
    at net.corda.node.services.statemachine.StateMachineManagerImpl.createFiber(StateMachineManagerImpl.kt:423)
    at net.corda.node.services.statemachine.StateMachineManagerImpl.createFiber$default(StateMachineManagerImpl.kt:422)
    at net.corda.node.services.statemachine.StateMachineManagerImpl$startFlow$fiber$1.invoke(StateMachineManagerImpl.kt:525)
    at net.corda.node.services.statemachine.StateMachineManagerImpl$startFlow$fiber$1.invoke(StateMachineManagerImpl.kt:63)
    at net.corda.nodeapi.internal.persistence.CordaPersistence.inTopLevelTransaction(CordaPersistence.kt:152)
    at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:138)
    at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:124)
    at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:131)
    at net.corda.node.services.statemachine.StateMachineManagerImpl.startFlow(StateMachineManagerImpl.kt:524)
    at net.corda.node.internal.FlowStarterImpl$startFlow$1.invoke(AbstractNode.kt:898)
    at net.corda.node.internal.FlowStarterImpl$startFlow$1.invoke(AbstractNode.kt:896)
    at net.corda.node.utilities.AffinityExecutor$fetchFrom$1.get(AffinityExecutor.kt:41)
    at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at net.corda.node.utilities.AffinityExecutor$ServiceAffinityExecutor$1$thread$1.run(AffinityExecutor.kt:62)
Caused by: java.lang.NoSuchFieldException: target
    at java.lang.Class.getDeclaredField(Class.java:820)
    at co.paralleluniverse.concurrent.util.ThreadAccess.<clinit>(ThreadAccess.java:48)
    ... 23 more
remmeier commented 5 years ago

to reproduce one can attempt to build https://github.com/corda/cordapp-example with https://adoptopenjdk.net/releases.html?variant=openjdk8&jvmVariant=openj9 (64-bit, jdk, java 8)

ipkpjersi commented 5 years ago

I'm not sure if that's related to this issue or not.

remmeier commented 5 years ago

yes, not the exactly same issue: ones a different kind of jdk, the other time restrictions of newer jdks. But in both cases caused by ThreadAccess doing rather JDK-implementation-specific reflection access.

ca-mu commented 5 years ago

I believe this warning is a result of the new Java module system. To get rid of this warning add: "--add-opens=java.base/java.lang=ALL-UNNAMED" This will work even when the illegal access default changes to "--illegal-access=deny" since you are explicitly enabling access. Let me know !!

ipkpjersi commented 5 years ago

While it may still work when --illegal-access gets changed to deny, future versions of Java will explicitly remove illegal reflective access altogether and it is quite likely this library will no longer work unless the code in question is explicitly changed. In other words, --add-opens is merely a temporary workaround.

ca-mu commented 5 years ago

I agree that the code should be changed, if possible, but I also thought that the "--add-opens" approach provided a permanent solution, since it is part of the module management system. If that is not the case, I'd like to learn about it and would greatly appreciate it if you could point me in the right direction.

bhartleyIBM commented 5 years ago

The OpenJ9 issue arises from the fact that there is no 'target' variable in the thread class. This is arguably a private field. I suspect it is actually a separate issue though. I believe it is worthy of remediation

ElazarNeeman commented 4 years ago

It seems that there are also issues with OpenJ9 in the same area (in this case I tested with the Java 8 flavor in the context of R3 Corda):

Caused by: java.lang.AssertionError: java.lang.NoSuchFieldException: target
  at co.paralleluniverse.concurrent.util.ThreadAccess.<clinit>(ThreadAccess.java:80)
  at co.paralleluniverse.fibers.Fiber.<init>(Fiber.java:199)
  at co.paralleluniverse.fibers.Fiber.<init>(Fiber.java:457)
  at net.corda.node.services.statemachine.FlowStateMachineImpl.<init>(FlowStateMachineImpl.kt:48)
  at net.corda.node.services.statemachine.StateMachineManagerImpl.createFiber(StateMachineManagerImpl.kt:423)
  at net.corda.node.services.statemachine.StateMachineManagerImpl.createFiber$default(StateMachineManagerImpl.kt:422)
  at net.corda.node.services.statemachine.StateMachineManagerImpl$startFlow$fiber$1.invoke(StateMachineManagerImpl.kt:525)
  at net.corda.node.services.statemachine.StateMachineManagerImpl$startFlow$fiber$1.invoke(StateMachineManagerImpl.kt:63)
  at net.corda.nodeapi.internal.persistence.CordaPersistence.inTopLevelTransaction(CordaPersistence.kt:152)
  at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:138)
  at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:124)
  at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:131)
  at net.corda.node.services.statemachine.StateMachineManagerImpl.startFlow(StateMachineManagerImpl.kt:524)
  at net.corda.node.internal.FlowStarterImpl$startFlow$1.invoke(AbstractNode.kt:898)
  at net.corda.node.internal.FlowStarterImpl$startFlow$1.invoke(AbstractNode.kt:896)
  at net.corda.node.utilities.AffinityExecutor$fetchFrom$1.get(AffinityExecutor.kt:41)
  at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at net.corda.node.utilities.AffinityExecutor$ServiceAffinityExecutor$1$thread$1.run(AffinityExecutor.kt:62)
Caused by: java.lang.NoSuchFieldException: target
  at java.lang.Class.getDeclaredField(Class.java:820)
  at co.paralleluniverse.concurrent.util.ThreadAccess.<clinit>(ThreadAccess.java:48)
  ... 23 more

I experience same problem in openJ9 my microservie does not loads any WA for this ?