Reactive Javalin with Kotlin coroutines

[dzikoysk]

At this moment it's just a quick follow up to the conversation on Discord about the async context and the natural expansion of non-blocking io that became an industry standard.

Benefits:

• Limit various huge thread-pools that impacts the memory footprint
• Simplify async requests by a providing a dedicated api

Candidates:

• Kotlin Coroutines (preferred, probably the best solution atm)
  • Pros: Should be really simple to implement, great support from Kotlin and its libraries
  • Cons: It's Kotlin, so Java apps can't really benefit from this feature. It should be probably provided in a dedicated module.
• RxJava
  • Pros: Works in every JVM based environment, it's known because of the Spring WebFlux
  • Cons: Quite a huge library, not as great as coroutines and I have no idea if it is hard to implement.
• Loom
  • Already implemented, but still unstable and requires whole dedicated JVM.

Some related conversations in the past:

• Inspired by Spring WebFlux support for project reactor
• Reactive Streams
• Async event-loop

[tipsy]

@dzikoysk this ticket is a bit different from what we discussed on Discord. It has always been my intention for Javalin to offer a simple and blocking model, which is sufficient for the majority of use cases. If your service has a couple of slow endpoints, or your are lucky enough to use a HTTP or DB library that supports futures, we have the ctx.future(future) method, which I think works fairly well (at last after the recent changes).

A general rewrite to be more async friendly is not something I think will be positive for the project. The simple change from ctx.result(future) to ctx.future(future) sparked a lot of discussion in different GitHub issues and on different chat servers, with a lot of confusion and a lot of people talking past each other. I think the project should stay simple and blocking, both for the sanity of the community, and it's primary maintainer (😅). I'm not knowledgeable enough about Java async to feel confident in providing a good solution.

The cons you listed for Kotlin Coroutines and RxJava are huge, so I will fight hard to keep this out of the core. If someone wants to make a module then that's still possible.

Personally I am a big fan of Loom, which is why Loom is enabled by default now (if it's present in the runtime). For people who need a mature and reactive setup on older Java versions I would rather point them to Vert.x.

All of this being said, I still see value in having a simple method lifting the request out of the request threadpool, so I'm very open to that.

[dzikoysk]

@tipsy First of all, I don't want to make you feel I'm pushing you to implement this :) As you already said, you don't have experience with this part of web development, so we need a common ground to discuss related topics. At this moment, the reactivity is just a magic blackbox for you. I understand also why you're a fan of Loom - you can just benefit from the concept without really exploring it. I'll try to a little describe the scenario and possible solution and if you're still sure you don't want to see it in the project, it probably means I failed in the explanation part.

About

this ticket is a bit different from what we discussed on Discord

It's not really different, because marking a HttpRequest as async and then processing it independently is exactly what nio stands for and where reactive solutions just simplifies the handling. By rejecting similar issues and refusing to at least partly (e.g. Kotlin only) support this concept, we're also no more lightweight web framework, as the resources consumption in a real world app is just much higher than in the used widely these days reactive alternatives.

Scenerio As an example, Jetty by default starts with thread-pool that may scale probably up to 200 threads. Now, by integrating various services like standard db, tasks let's why have around another 100 threads. We already wasted more than 250MB of memory just to keep these OS level threads and where most of them are just blocked awaiting for response. Of course, that's the case when the app really do something, in a test env the app doesn't consume it, because there is no real traffic. In a contrast, standard reactive microservice uses fixed thread-pool around 16 threads for the whole application.

In general

• Blocking server: good for CPU-heavy operations where we can maximize the performance gained by running task in a dedicated thread. In case that we don't really use CPU, it just wastes a lot of resources and makes async programming literally a pain by handmade implementations and utilities of concurrent systems. That's just bad for everyone.
• Non-blocking server (reactive): Slower, because of thread context switching and various mechanisms, but thanks to the improved architecture, we can use make a better use of created threads.

Web developers realized that the standard server in most cases just awaits for a result from another source rather than uses CPU and do something heavy. These days we use microservices deployed to the cloud, so we just want to limit resources consumption where we can.

Assumptions

• The implementation has to be simple, a few lines of code that just delegates the processing to another place.
• The implementation should not affect any of the existing methods.
  • Just one new method in API or none in case of some kind of a auto detection

Solution I'll talk about coroutines as I'm quite sure an implementation is even simpler than the current future api implementation. I also don't really now about the internals of RxJava and how to boot it up. I guess it might be similar to submitting a task to a dedicated ExecutorService (like in Loom) 🤔

What is suspended function? It's a built-in into the language modifier that marks method as a method that might be run into the async context:

suspend fun doSomething() { } // that's all, as user you don't really have to know anything more about

The usage might look like this:

suspend fun handle(ctx: Context) {
  // findAll is also suspend, so there is a suspend call chain and Kotlin handles it
  // we don't need to do anything more, Kotlin is going to manage this automatically under the hood
  val result = mongoRepository.findAll()
}

We can also provide .async method dedicated for suspend functions. It might be registered using Kotlin extension functions, so the Java users won't even see this in their api.

fun handle(ctx: Context) {
  ctx.async { // async starts , also suspend
    val result = mongoRepository.findAll()
  }
}

To launch such a method we just need to create a dispatcher that is based on some ExecutorService:

httpRequest.startAsync()

launch(dispatcher) {
  handle.invoke(ctx)
}

That's pretty much all, user can now limit the thread-pool assigned to Jetty to let's say 16 threads (based that he has 16 logical cpus) and then set up the dispatcher size also to 16 threads. The app now runs on 32 threads and the outcome should be similar to the app that runs on 300 threads. Of course, to compare exact values we'd need a benchmark, but it should be something around these values.

Summary In this section I'll rather talk about my opinion on the topic and personal preferences, as the content above was meant to be "objective" - as far as possible of course.

The simple change from ctx.result(future) to ctx.future(future) sparked a lot of discussion in different GitHub issues and on different chat servers, with a lot of confusion and a lot of people talking past each other

I don't really think it's really related. That's confusing, because it changes existing API. It also still bases on CompletableFutures that are great utility to say that "we don't have a value yet", but it doesn't really change the handling process and its cost.

Personally I am a big fan of Loom, which is why Loom is enabled by default now (if it's present in the runtime).

It sounds great, but it's still unstable and we don't have any guarantee about it, so it's unusable in production environments. Another thing is, do you really want to submit any non-marked as async code having a promise it'll work properly. Using various languages and libraries we always have to explicitly define if it should be run in async context. It's just a loose thought about "silent fibers", not really related to the topic of this issue. 😳

RxJava are huge, so I will fight hard to keep this out of the core

Finally, speaking once again about RxJava. I'm not quite a huge fan of this project, because it really modifies the way how you can develop your business logic and in fact, it requires a lot of experience do it properly. In this area I'd agree that Javalin is not a good place to implement this, because we both can agree that nobody who already is able to use it and have a strong need to this, should pick Javalin. I've mentioned it as it's just one of the available and widely used frameworks that delivers reactive functionalities.

That's all for now I think, I hope it removes the magic aspect of the implementation and as you see, it should be even easier to support than the good api for futures which was kinda problematic for a while.

To be honest, as I'm mainly talking about the coroutines, it might be good idea to a little modify the title as this moment it is probably the only way we could see reactivity in Javalin 😅

[rbygrave]

Using various languages and libraries we always have to explicitly define if it should be run in async context.

Not with Loom no. The JVM runtime internally knows where a virtual thread yields and does it. That is, as part of loom the jdk internals have been modified to yield at all the places needed such as IO, lock, Thread.sleep() etc [with the caveat around the use of sychronized]. So like golang / goroutines.

The app now runs on 32 threads and the outcome should be similar to the app that runs on 300 threads.

Only if it is reactive all the way. So for example we can not use jdbc and need to change to use something like r2dbc (or perhaps use a store that provides a reactive driver like mongo as in your examples above). As a side note, for myself I've looked pretty closely at r2dbc and for example their own benchmark does not have it as fast as jdbc. r2dbc vs jdbc is almost another full separate topic but ultimately if a stack is using jdbc then it is not going to be "reactive all the way".

The app now runs on 32 threads

We get exactly this same effect with Loom with arguably some benefits:

• We just need to start using ExecutorService based on virtual threads (so basically no change to existing apps)
• We can use any and all existing libraries including JDBC drivers (with a caveat around synchronized keyword wrapping IO).
• For debugging, try catch, stack traces we get the same simplicity we have today (and seems like the main reasons some people revert back from reactive stacks)
• Anything that uses ThreadLocal like MDC still work
• AOP style things like @Transactional and @Timed still work
• Maybe virtual thread park/unpark cost is cheaper than thread context switch cost (this is expected as part of Loom but unproven yet?)

If you are in the reddit r/java community you would have seen there has been a lot of discussion on Loom and reactive frameworks like Webflux and Vertx. What is rather interesting is the number of people who have tried spring webflux etc and then decided to go back. It does not seem to be 1 way adoption if that sample is accurate.

Anyway, one of the more recent discussions: https://www.reddit.com/r/java/comments/m3n0jw/will_loom_make_vertx_rxjava_and_event_loop_based/

It sounds great, but it's still unstable and we don't have any guarantee about it

To me unstable is the wrong word. I've been playing around with the last few EA releases and it's been perfectly stable. I'm using the latest EA release for a Loom demo tomorrow and it is pretty darn impressive. Sure, it almost certainly won't be in the officially released 17 but 18 might be realistic and that is maybe 8 months away?

kotlin suspended function

To me this comes under the "coloured function problem" (https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/). Java (like golang) remains a language without methods marked/annotated/indicated as being async (unlike C#, javascript and kotlin coroutines).

Ultimately it depends on what time frame a project works to in terms of stack choice and API design and if you believe Loom makes all the reactive libraries somewhat legacy (including kotlin coroutines).

I definitely recommend trying loom if you haven't already.

[dzikoysk]

All you've listed here is more like a follow up to the Loom references in my thread. I've tested it and various other projects as I'm a quite a huge fan of these researches - not gonna lie that Valhalla is my favorite. Anyway, it is not a production ready solution for now, it's still a choice for people who host their apps locally and may agree on some consequences caused by their choices. My proposal is not against Loom, but it's directed to people who don't want to build the entire application around Loom, because of various reasons.

Only if it is reactive all the way. [...]

That's true, but it's not related to this issue. User has to cover this cases, not the maintainer of Javalin.

We get exactly this same effect with Loom with arguably some benefits

Of course they're, native fibers are just great and doesn't require some kind of stub abstraction layer created on top of the language that does not support such a features.

What is rather interesting is the number of people who have tried spring webflux etc and then decided to go back. It does not seem to be 1 way adoption if that sample is accurate.

Many people tried many technologies and they didn't liked it. The fact about RxJava and reactive programming in general is that the topic is really hard and requires a lot of experience to do it right. As I said in my previous comment - I don't think that Javalin is the best place to implement this anyway.

Coloured function problem [...] Ultimately it depends on what time frame a project works to in terms of stack choice and API design and if you believe Loom makes all the reactive libraries somewhat legacy

In general, all references to the technology that is not widely used are based on a believe and hope. It might be even a case for Kotlin, because the belief that someday Java will catch-up the functionalities that Kotlin provides (and it is happening right now in the official Java releases), will just make it redundant. I'm talking about people that require the solution now and the existing possibilities are just fine for them.

In theory, let's say that Loom might be merged into the official Java releases and we're sure it won't be even a case in the next LTS (17), but rather around 19. It means that in a next few years people still won't be able to use this in this project, that's kinda sad. Even if we'll finally see Loom in Java 19, it doesn't mean that the whole community will use Java 19. Various teams are just fine by using older LTS releases for a while.

That was a perspective of end-users, what about library maintainers and so on. Loom won't make them disappear, it might just reduce their audience in the future.

In the end, why do we assume that proposed coroutines are against fibers. Probably that's the most painless way to write that reactive-like code these days. The dispatcher may use Loom's ExecutorService as well in the future.

[mattwelke]

I think the project should stay simple and blocking, both for the sanity of the community, and it's primary maintainer (😅). I'm not knowledgeable enough about Java async to feel confident in providing a good solution.

Just wanted to chime in and mention that being a newbie to Java, coming from Node.js and Go where I didn't have to think very much about concurrency, I really like Javalin because of how easy it is for me to understand. I appreciate the effort that's been put into keeping the framework simple and lightweight when it comes to programming model (even if it isn't "lightweight" when it comes to efficient use of OS resources, as pointed out by @dzikoysk). I like that it's sticking around and fulfilling that niche.

[dzikoysk]

@mattwelke this proposal is not against any of the value represented by Javalin, I'd say it's just another area to cover as a simple enhancement to simplify async usage and related scenarios using a tiny layer.

[dzikoysk]

As a research and some kind of PoC for this thread, I'm currently during tests of the one of javalin-rfcs module that provides simple routing based on coroutines:

• https://github.com/reposilite-playground/javalin-rfcs

The coroutines support is developed as a plugin and does not require modified version of Javalin. The API is quite stable at this moment, so I decided to use it in a real-world app, so on, I'm implementing this as a part of my open source project (on experimental branch, as well as the rewritten OpenApi plugin that uses annotation-processor):

• https://github.com/dzikoysk/reposilite/tree/3.0

More details about progress and results I share on Javalin's Discord server.

[mattwelke]

@dzikoysk This is really cool and I think it might be worth using Discussions on GitHub for the details, that way people who aren't on the Discord or who don't know where to find the conversations on Discord can see it here.

@tipsy Thoughts on adding Discussions to this repo?

[dzikoysk]

@mattwelke Thanks :) I don't quite like GH Discussions as I feel it's more for projects that really need to move away "spam" related to questions about framework and how to use it rather than topics about its development. I also use mostly Discord to communicate on various servers, so I prefer to post it there.

Speaking of the issue, I don't think this feature will be a part of official Javalin as I don't see the interest in this topic, probably because of the narration around Loom and how it solves related conversations in the further years without changing anything.

I'm quite sure that even if I fully test my reactive-routing plugin it's just not possible to merge it in the current form. This plugin somehow solves problems raised by #1267 and #1278 and proposes an abstraction on top of that. To be honest, the builtin implementation could be much more simple, probably just a few lines, but its design should be a part of conversation where contributors agree that it's something they want to implement. As we can see, it's more like a fun fact and there is no involvement in this area, so I guess it just won't happen. That's ok, luckily it's possible to use it as a plugin what's enough for me :)

[mattwelke]

Does the Javalin project have a concept of official plugins? Your plugin may deserve more legitimacy than merely being a 3rd party plugin. You're right that there isn't a huge interest in Kotlin coroutine Javalin, but there is some interest. And I bet that among those who are interested, they'd prefer using an official plugin if they could.

[dzikoysk]

Kind of through https://github.com/javalin. The organization may find more attention when #1297 will happen. Anyway, that's still part of my research as a part of Reposilite project and its playground repository as mentioned above, so I don't think it could be used as an official plugin - it uses, and probably will use even more, various non-common extensions such as e.g. expressible to provide better experience in developing an app that I can relay on with a pretty straightforward implementation. Speaking of the official plugin, I think it should limit external dependencies to the minimum and provide only the functionality it stands for.

[tipsy]

@mattwelke Does the Javalin project have a concept of official plugins?

No, but it's something I want to look into after releasing Javalin 4.

@mattwelke Thoughts on adding Discussions to this repo?

I agree with @dzikoysk here, I think issues (or discord) are better than Discussions. We don't have the amount of traffic that justifies introducing it, IMO.

[asad-awadia]

the ctx.result(future) already allows using suspending functions using the jdk future coroutine library

also, one of the best parts about javalin is its simple blocking nature -- if i wanted a tonne of async - I would just use vertx instead

[dzikoysk]

the ctx.result(future) already allows using suspending functions using the jdk future coroutine library

As I said before, this is what we're currently testing with plugin api - javalin-rfcs :: javalin-reactive-routing. Unfortunately only http routes can be covered with this, as other handlers - such as before & after, does not support async requests (even if it is exposed in Context in these routes). Another topic is that even ctx.resultFuture() was not thread-safe for like 3 years (#1385) and still has some problems (discussion on Javalin's discord server).

also, one of the best parts about javalin is its simple blocking nature -- if i wanted a tonne of async - I would just use vertx instead

Introducing any of the proposed changes does not affect blocking context, so referring that in this thread is kinda pointless. I could somehow agree with this statement if it would be a conversation about introducing async context at the first place, but as we can see it's available for a while and people use it so there is an area to provide enhancements. There is also a slightly difference between building a fully reactive engine and providing a partial support to just allow people to create such a context.

Anyway, also I said, I'm quite sure we won't see such enhancements to the core project, but I'm still for improving the current implementation to provide/simplify the possibility of introducing such a support through plugin api.

[dzikoysk]

Experimental coroutines-based routing implementation is now available as a submodule of javalin-routing-extensions plugin. Example usage:

• https://github.com/javalin/javalin-routing-extensions#coroutines