quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.84k stars 2.7k forks source link

Investigate coroutines #1957

Open FroMage opened 5 years ago

FroMage commented 5 years ago
FroMage commented 5 years ago

OK, so it was not easy, and this is all still very experimental, but:

[INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] Running Quarkus native-image plugin on Java HotSpot(TM) 64-Bit Server VM
[INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] /home/stephane/no-backup/src/graalvm-ce-1.0.0-rc14/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar quarkus-integration-test-hibernate-orm-panache-999-SNAPSHOT-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:+PrintAnalysisCallTree -H:-AddAllCharsets -H:EnableURLProtocols=http,https --enable-all-security-services -H:NativeLinkerOption=-no-pie -H:-SpawnIsolates -H:+JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace
[quarkus-integration-test-hibernate-orm-panache-999-SNAPSHOT-runner:31060]    classlist:  18,729.38 ms
[quarkus-integration-test-hibernate-orm-panache-999-SNAPSHOT-runner:31060]        (cap):   1,246.61 ms
[quarkus-integration-test-hibernate-orm-panache-999-SNAPSHOT-runner:31060]        setup:   2,585.64 ms
17:40:29,375 INFO  [org.hib.Version] HHH000412: Hibernate Core {5.4.2.Final}
17:40:29,403 INFO  [org.hib.ann.com.Version] HCANN000001: Hibernate Commons Annotations {5.1.0.Final}
17:40:29,438 INFO  [org.hib.dia.Dialect] HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
17:40:31,309 INFO  [com.arj.ats.arjuna] ARJUNA012170: TransactionStatusManager started on port 45635 and host 127.0.0.1 with service com.arjuna.ats.arjuna.recovery.ActionStatusService
17:40:32,017 INFO  [org.xnio] XNIO version 3.7.0.Final
17:40:32,135 INFO  [org.xni.nio] XNIO NIO Implementation Version 3.7.0.Final
17:40:34,124 INFO  [org.jbo.threads] JBoss Threads version 3.0.0.Alpha4
QUASAR WARNING: Quasar Java Agent isn't running. If you're using another instrumentation method you can ignore this message; otherwise, please refer to the Getting Started section in the Quasar documentation.
[quarkus-integration-test-hibernate-orm-panache-999-SNAPSHOT-runner:31060]     analysis:  23,730.77 ms
Printing call tree to /home/stephane/src/java-eclipse/quarkus/integration-tests/hibernate-orm-panache/target/reports/call_tree_quarkus-integration-test-hibernate-orm-panache-999-SNAPSHOT-runner_20190409_174100.txt
Printing list of used classes to /home/stephane/src/java-eclipse/quarkus/integration-tests/hibernate-orm-panache/target/reports/used_classes_quarkus-integration-test-hibernate-orm-panache-999-SNAPSHOT-runner_20190409_174102.txt
Printing list of used packages to /home/stephane/src/java-eclipse/quarkus/integration-tests/hibernate-orm-panache/target/reports/used_packages_quarkus-integration-test-hibernate-orm-panache-999-SNAPSHOT-runner_20190409_174102.txt
Error: unsupported features in 2 methods
Detailed message:
Error: Exception handler can be reached by both normal and exceptional control flow
Call path from entry point to com.github.fromage.quasi.strands.dataflow.Val$1.run(): 
    at com.github.fromage.quasi.strands.dataflow.Val$1.run(Val.java)
    at com.github.fromage.quasi.strands.SuspendableUtils$VoidSuspendableCallable.run(SuspendableUtils.java:45)
    at com.github.fromage.quasi.strands.SuspendableUtils$VoidSuspendableCallable.run(SuspendableUtils.java:33)
    at com.github.fromage.quasi.fibers.Fiber.run(Fiber.java:1100)
    at com.github.fromage.quasi.fibers.Fiber.run1(Fiber.java:1095)
    at com.github.fromage.quasi.fibers.Fiber.exec(Fiber.java:791)
    at com.github.fromage.quasi.fibers.FiberForkJoinScheduler$FiberForkJoinTask.exec1(FiberForkJoinScheduler.java:272)
    at com.github.fromage.quasi.concurrent.forkjoin.ParkableForkJoinTask.doExec(ParkableForkJoinTask.java:118)
    at com.github.fromage.quasi.concurrent.forkjoin.ParkableForkJoinTask.exec(ParkableForkJoinTask.java:75)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
    at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:473)
    at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
    at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)
--------------------------------------------------------------------------------------------
-- WARNING: The above stack trace is not a real stack trace, it is a theoretical call tree---
-- If an interface has multiple implementations SVM will just display one potential call  ---
-- path to the interface. This is often meaningless, and what you actually need to know is---
-- the path to the constructor of the object that implements this interface.              ---
-- Quarkus has attempted to generate a more meaningful call flow analysis below          ---
---------------------------------------------------------------------------------------------

Possible path to com.github.fromage.quasi.strands.dataflow.Val$1.run():v
    com.github.fromage.quasi.strands.dataflow.Val$1.run():v
This is an implementation of com.github.fromage.quasi.strands.SuspendableRunnable printing path to constructors of com.github.fromage.quasi.strands.dataflow.Val$1

Possible path to com.github.fromage.quasi.strands.dataflow.Val$1.<init>(com.github.fromage.quasi.strands.dataflow.Val, com.github.fromage.quasi.strands.SuspendableCallable):v
    com.github.fromage.quasi.strands.dataflow.Val$1.<init>(com.github.fromage.quasi.strands.dataflow.Val, com.github.fromage.quasi.strands.SuspendableCallable):v
    com.github.fromage.quasi.strands.dataflow.Val.<init>(com.github.fromage.quasi.fibers.FiberScheduler, com.github.fromage.quasi.strands.SuspendableCallable):v
    com.github.fromage.quasi.strands.dataflow.Val.<init>(com.github.fromage.quasi.strands.SuspendableCallable):v
    com.github.fromage.quasi.strands.dataflow.Val.<init>():v
    com.github.fromage.quasi.fibers.Fiber.<init>(java.lang.String, com.github.fromage.quasi.fibers.FiberScheduler, int, com.github.fromage.quasi.strands.SuspendableCallable):v
    com.github.fromage.quasi.fibers.Fiber.<init>(java.lang.String, int, com.github.fromage.quasi.strands.SuspendableCallable):v
    com.github.fromage.quasi.fibers.Fiber.<init>(java.lang.String, int, com.github.fromage.quasi.strands.SuspendableRunnable):v
    com.github.fromage.quasi.fibers.Fiber.<init>(com.github.fromage.quasi.strands.SuspendableRunnable):v
    io.quarkus.example.panache.TestEndpoint.fiber(com.github.fromage.quasi.strands.SuspendableCallable):j
    io.quarkus.example.panache.TestEndpoint.getAsync():j
    io.quarkus.example.panache.TestEndpoint.getAsync():j
    com.oracle.svm.reflect.TestEndpoint_getAsync_87e7cd4eaafc86e2d3f3ee6267ed235c8b29fb4a.invoke(java.lang.Object, java.lang.Object[]):j
This is an implementation of sun.reflect.MethodAccessor printing path to constructors of com.oracle.svm.reflect.TestEndpoint_getAsync_87e7cd4eaafc86e2d3f3ee6267ed235c8b29fb4a

Original exception that caused the problem: org.graalvm.compiler.core.common.PermanentBailoutException: Exception handler can be reached by both normal and exceptional control flow
    at org.graalvm.compiler.java.BciBlockMapping.addSuccessor(BciBlockMapping.java:763)
    at org.graalvm.compiler.java.BciBlockMapping.iterateOverBytecodes(BciBlockMapping.java:614)
    at org.graalvm.compiler.java.BciBlockMapping.build(BciBlockMapping.java:517)
    at org.graalvm.compiler.java.BciBlockMapping.create(BciBlockMapping.java:1109)
    at org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:807)
    at org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:785)
    at org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:95)
    at org.graalvm.compiler.phases.Phase.run(Phase.java:49)
    at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
    at org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
    at org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
    at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:213)
    at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:332)
    at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310)
    at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300)
    at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107)
    at com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:191)
    at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:352)
    at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:394)
    at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:508)
    at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:169)
    at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Error: Exception handler can be reached by both normal and exceptional control flow
Call path from entry point to io.quarkus.example.panache.TestEndpoint.lambda$fiber$48e7e753$1(SuspendableCallable, CompletableFuture): 
    at io.quarkus.example.panache.TestEndpoint.lambda$fiber$48e7e753$1(TestEndpoint.java)
    at io.quarkus.example.panache.TestEndpoint$$Lambda$579/1565660052.run(Unknown Source)
    at com.github.fromage.quasi.strands.SuspendableUtils$VoidSuspendableCallable.run(SuspendableUtils.java:45)
    at com.github.fromage.quasi.strands.SuspendableUtils$VoidSuspendableCallable.run(SuspendableUtils.java:33)
    at com.github.fromage.quasi.fibers.Fiber.run(Fiber.java:1100)
    at com.github.fromage.quasi.fibers.Fiber.run1(Fiber.java:1095)
    at com.github.fromage.quasi.fibers.Fiber.exec(Fiber.java:791)
    at com.github.fromage.quasi.fibers.FiberForkJoinScheduler$FiberForkJoinTask.exec1(FiberForkJoinScheduler.java:272)
    at com.github.fromage.quasi.concurrent.forkjoin.ParkableForkJoinTask.doExec(ParkableForkJoinTask.java:118)
    at com.github.fromage.quasi.concurrent.forkjoin.ParkableForkJoinTask.exec(ParkableForkJoinTask.java:75)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
    at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:473)
    at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
    at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)
--------------------------------------------------------------------------------------------
-- WARNING: The above stack trace is not a real stack trace, it is a theoretical call tree---
-- If an interface has multiple implementations SVM will just display one potential call  ---
-- path to the interface. This is often meaningless, and what you actually need to know is---
-- the path to the constructor of the object that implements this interface.              ---
-- Quarkus has attempted to generate a more meaningful call flow analysis below          ---
---------------------------------------------------------------------------------------------

Possible path to io.quarkus.example.panache.TestEndpoint.lambda$fiber$48e7e753$1(com.github.fromage.quasi.strands.SuspendableCallable, java.util.concurrent.CompletableFuture):v
    io.quarkus.example.panache.TestEndpoint.lambda$fiber$48e7e753$1(com.github.fromage.quasi.strands.SuspendableCallable, java.util.concurrent.CompletableFuture):v
    io.quarkus.example.panache.TestEndpoint$$Lambda$1402d16eacdf9d5d14ca78e42cd2448f1dfc9ea0.run():v
This is an implementation of com.github.fromage.quasi.strands.SuspendableRunnable printing path to constructors of io.quarkus.example.panache.TestEndpoint$$Lambda$1402d16eacdf9d5d14ca78e42cd2448f1dfc9ea0

Possible path to io.quarkus.example.panache.TestEndpoint$$Lambda$1402d16eacdf9d5d14ca78e42cd2448f1dfc9ea0.<init>(com.github.fromage.quasi.strands.SuspendableCallable, java.util.concurrent.CompletableFuture):v
    io.quarkus.example.panache.TestEndpoint$$Lambda$1402d16eacdf9d5d14ca78e42cd2448f1dfc9ea0.<init>(com.github.fromage.quasi.strands.SuspendableCallable, java.util.concurrent.CompletableFuture):v
    io.quarkus.example.panache.TestEndpoint$$Lambda$1402d16eacdf9d5d14ca78e42cd2448f1dfc9ea0.get$Lambda(com.github.fromage.quasi.strands.SuspendableCallable, java.util.concurrent.CompletableFuture):c
    io.quarkus.example.panache.TestEndpoint.fiber(com.github.fromage.quasi.strands.SuspendableCallable):j
    io.quarkus.example.panache.TestEndpoint.getAsync():j
    io.quarkus.example.panache.TestEndpoint.getAsync():j
    com.oracle.svm.reflect.TestEndpoint_getAsync_87e7cd4eaafc86e2d3f3ee6267ed235c8b29fb4a.invoke(java.lang.Object, java.lang.Object[]):j
This is an implementation of sun.reflect.MethodAccessor printing path to constructors of com.oracle.svm.reflect.TestEndpoint_getAsync_87e7cd4eaafc86e2d3f3ee6267ed235c8b29fb4a

Original exception that caused the problem: org.graalvm.compiler.core.common.PermanentBailoutException: Exception handler can be reached by both normal and exceptional control flow
    at org.graalvm.compiler.java.BciBlockMapping.addSuccessor(BciBlockMapping.java:763)
    at org.graalvm.compiler.java.BciBlockMapping.iterateOverBytecodes(BciBlockMapping.java:614)
    at org.graalvm.compiler.java.BciBlockMapping.build(BciBlockMapping.java:517)
    at org.graalvm.compiler.java.BciBlockMapping.create(BciBlockMapping.java:1109)
    at org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:807)
    at org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:785)
    at org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:95)
    at org.graalvm.compiler.phases.Phase.run(Phase.java:49)
    at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
    at org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
    at org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
    at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:213)
    at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:332)
    at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310)
    at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300)
    at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107)
    at com.oracle.graal.pointsto.flow.StaticInvokeTypeFlow.update(InvokeTypeFlow.java:346)
    at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:508)
    at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:169)
    at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception

Which is the cause of the issue I was talking about earlier, with generated exception tables such as:

      Exception table:
         from    to  target type
            45   137   162   Class com/github/fromage/quasi/fibers/SuspendExecution
            45   137   162   Class com/github/fromage/quasi/fibers/RuntimeSuspendExecution
            45   137   137   Class java/lang/reflect/UndeclaredThrowableException
            48    53   120   Class java/lang/Throwable // THIS ONE
           105   117   120   Class java/lang/Throwable // AND THIS ONE
            45   137   153   any

Where both try blocks point to the same catch block. We will need to tweak Quasar to duplicate the try blocks.

FroMage commented 5 years ago

You can take this over in https://github.com/FroMage/quarkus/tree/quasi-hacking @dmlloyd

FroMage commented 5 years ago

BTW, the biggest differences IMO between Quarkus are:

dmlloyd commented 5 years ago

It's really interesting/strange that SVM complains about this; I wonder if they make some special assumptions. But yeah theoretically it should be possible to duplicate the exception handler blob.

FroMage commented 5 years ago

Ah yes, right, it's the handler we need to duplicate, not the try block, of course :)

FroMage commented 5 years ago

So I tried EA-Async in https://github.com/FroMage/quarkus/tree/ea-async and it Just Works™.

With the caveats that I mentioned earlier, but still.

FroMage commented 5 years ago

Ferget the other branch: use https://github.com/FroMage/quarkus/tree/coroutines to get quarkus-coroutines-ea-async and quarkus-coroutines-quasar modules.

FroMage commented 5 years ago

Well, I tried ea-coroutines in panache-rx and that turned out pretty amazing: https://github.com/FroMage/quarkus/commit/cbe1aca25f4d876b343968ef6e474124bde67a44?diff=split

FroMage commented 5 years ago

Ah no, sometimes ea-async does not work in SVM:

Error: Non-reducible loop
Call path from entry point to io.quarkus.example.panache.TestEndpoint.async$testRxModel(TestEndpoint, CompletionStage, List, RxPerson, long, RxPerson, RxPerson, int, int, Object): 
    at io.quarkus.example.panache.TestEndpoint.async$testRxModel(TestEndpoint.java)
    at io.quarkus.example.panache.TestEndpoint$$Lambda$589/1206642173.apply(Unknown Source)
    at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952)
    at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
    at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
    at org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612)
    at org.xnio.nio.WorkerThread.run(WorkerThread.java:479)
    at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:473)
    at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
    at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)
FroMage commented 5 years ago

DAMNIT.

FroMage commented 5 years ago

Appears to be similar to https://github.com/oracle/graal/issues/366 because I can get around it by inlining my loop (as a workaround).

cromefire commented 4 years ago

Coroutines should work well now with 20.1 Also #10162 seems to be a (active) duplicate of this

ambition-consulting commented 4 years ago

@FroMage Any plans on retesting with GraalVM 2.1?

FroMage commented 4 years ago

Not ATM, no, I've no time. But if you want to try, let us know :)

FroMage commented 3 years ago

I tested with 20.3.1 and I still get the same issue. It looks like they didn't fix it in general. Perhaps they special-cased Kotlin patterns, or Kotlin changed their implementation?

Error: Exception handler can be reached by both normal and exceptional control flow
Call path from entry point to com.example.Fibers.lambda$fiber$d718b233$1(SuspendableCallable, CompletableFuture): 
    at com.example.Fibers.lambda$fiber$d718b233$1(Fibers.java)
    at com.example.Fibers$$Lambda$644/0x00000007c0ec1440.run(Unknown Source)
dmlloyd commented 3 years ago

It's interesting that they care whether an exception handler is reachable via normal control flow. Especially since the distinction between an exception handler and normal control flow is so fine: for example, what if my exception handler were simply a GOTO to normal program execution? Surely that must be allowed, since empty catch blocks seem to function just fine.

I wonder whether it would be worth figuring out a change to the Quasar code to vector the exception code through a GOTO, so that there is never a GOTO (or fall-through) with the same destination as an exception handler. Maybe in conjunction with allocating a local variable for the exception so that the code doesn't get somehow folded by GraalVM back into the construct that was invalid in the first place.

FroMage commented 3 years ago

At the time, we thought about it, or perhaps a variation such as duplicating the exception blocks. But yeah, moving it out and using a GOTO could work, but also means having to pass access to local variables, some of which are not mutable and may require mutation.

At the time, we though that was too much trouble to fix. I still feel like, at least for me, it'd be a rabbit hole full of traps.

AlexMog commented 3 years ago

Hello, Don't know if this subject is still in progress or if tascalate async/await was mentioned here, but it should be usable to add async/await pattern to Quarkus I assume: https://github.com/vsilaev/tascalate-async-await (and, unlike EA Async, it is still maintained)