Open ikhoon opened 1 year ago
As a follow-up, I think running decorators from virtual threads would also be useful and a widely sought feature.
We may also want to consider either
1) Finding a way to set the workerGroup
to a virtual thread executor so that decorators are also run from virtual threads
2) Providing a way to execute from a blocking executor from the HttpServerHandler
(before the first decorator is triggered)
Considering Java 21 has been released pretty recently, I think this would be a cool feature to have now.
Does having virtual threads mean better performance for blocking tasks? Especially for blocking task executor?
As a follow-up, I think running decorators from virtual threads would also be useful and a widely sought feature.
Does that mean decorators will finally run outside the event loop while having a similar performance?
Hey thanks, we're definitely discussing designs on this at the moment.
Does having virtual threads mean better performance for blocking tasks? Especially for blocking task executor?
Currently, blockingTaskExecutor
only schedules requests to the blocking executor for specific types of services (i.e. grpc, annotated). We would like to serve requests and callbacks from virtual threads entirely. (including decorators)
Does that mean decorators will finally run outside the event loop while having a similar performance?
That's the goal. Ideally, we will introduce a parameter which either accepts an (EventLoopGroup
|| virtual thread executor) so that users can choose between blocking and reactive style.
Ideally, we will introduce a parameter which either accepts an (EventLoopGroup || virtual thread executor) so that users can choose between blocking and reactive style.
I guess that would be much bigger effort?
@ikhoon's proposal in the description is probably just switching to virtual threads from platform threads for the blocking task executor when applicable ( i.e if Armeria is running on JDK 21+) ?
I guess that would be much bigger effort?
Right, we need many changes to fully support virtual threads. Because we don't know when we will reach that goal, this issue would be a minimal effort to serve user services on the virtual threads.
It would be a very useful improvement for us to make all processing possible via virtual threads. Not only for consistency and completeness of stack traces when using decorators (we are using a blocking task executor), but would also allow us to test our distributed system efficiently following this approach: https://jbaker.io/2022/05/09/project-loom-for-distributed-systems/
Would "thread pinning" be a concern?
https://www.infoworld.com/article/3713220/java-virtual-threads-hit-with-pinning-issue.html
Thread pinning will be fixed in the next Java version. https://mail.openjdk.org/pipermail/loom-dev/2024-February/006433.html
We have been spending a lot of time implementing LINE's internal requirements such as xDS protocol and so on. We didn't have time to look into this issue in detail.
I knew many people are looking forward to using this feature. Let me try to upgrade the build JDK version to 21 and use virtual threads as an experimental blocking task executor in the next version.
Thread pinning will be fixed in the next Java version. https://mail.openjdk.org/pipermail/loom-dev/2024-February/006433.html
We have been spending a lot of time implementing LINE's internal requirements such as xDS protocol and so on. We didn't have time to look into this issue in detail.
I knew many people are looking forward to using this feature. Let me try to upgrade the build JDK version to 21 and use virtual threads as an experimental blocking task executor in the next version.
By next Java version, which I suppose is Java 22 and that would not be an LTS.
I would suggest that you folks should wait for it to settle first before taking the leap. Without VirtualThread, the current Armeria is already doing more than what I can asked for.
If Project Loom is activated on the JRE, we may use
Thread.ofVirtual().factory()
to create virtual threads forBlockingTaskExecutor
instead. https://github.com/line/armeria/blob/d3c3811d7367e67ed4b7df47aa9fad5889948aed/core/src/main/java/com/linecorp/armeria/common/util/BlockingTaskExecutorBuilder.java#L131-L135Virtual threads require a small number of threads. Users will have less work to tune thread counts with less memory footprint.