V7 - Replacing Undertow Strategy

[patrickjp93]

Overview of the feature request

As per the V7 Roadmap Discussion, it seems to be a good time to swap the HTTP Server provider from Undertow to something else given IBM/Redhat do not support undertow anymore (apparently not true as per @jdubois' reply). The straightforward pick to replace it is Tomcat, as it is more or less the Spring Boot default.

In an email to @jdubois, I asked if it may be appropriate to have a broader discussion on HTTP Server options rather than pick Tomcat purely by ease/laziness/default.

This is not an exhaustive list, but some of the most well known competing providers are: Tomcat (Apache) Glassfish (Oracle) Jetty (Eclipse Foundation) Netty (Independent)

And beyond this are also some frameworks such as Comsat Web Actors which could be integrated for performance/resilience reasons, or at least documented for people who want to squeeze out the last 5-10% of their machines.

This is the most current set of benchmarks I could find, from the lovely people at DZone. It's everything from the last generation (Tomcat V8, with 10 about to replace 9). Documentation of EC2 instances, optimization flags, code, and other relevant details is succinct and fairly comprehensive.

Primary Goal: an informed discussion on choosing a new HTTP Server provider, paying attention to the computing landscape in respect to performance, security, and other concerns.

Motivation for or Use Case

JHipster should be positioned competitively with the best default infrastructure possible. The less users have to change, the more they'll use it.

Related issues or PR

10958

I don't have any EC2 instances or Azure/GCP boxes open at present, but if no one else has some spare machines for benchmarking the latest, I can probably take a crack at it in the midst of the COVID-19 lockdown. I'll just need some assistance with using HTTPS/setting up SSL certs correctly on a non-VPC, non-DNS environment.

• [x] Checking this box is mandatory (this is just to show you read everything)

[gmarziou]

Performance is one aspect, security is another one.

By choosing Tomcat, we rely on a solution which is wildly known and supported either as embedded or standalone. For example, it's much easier to find CIS template for Tomcat than for any other server, it's important when you build a PCI-DSS compliant solution.

[patrickjp93]

Jetty templates exist just as Tomcat ones do. My previous employer had several PCI-DSS compliant systems built on Jetty, reason being we could cut server upgrade costs by over 40%. If we wanted to target the easiest server to harden, you'd pick Glassfish. And Jetty is both standalone and embedded capable.

I have no dog in the fight, so I'm going to try to avoid appearing to shill for Jetty. I'm just after a broader, recorded discussion on alternatives.

[pascalgrimaud]

I'd vote for Tomcat, as it will follow our policy 1 -> https://www.jhipster.tech/policies/

We migrated to Undertow during v3 few years ago, at the same time than Microservices, and for performance reason.

[jdubois]

• I've had a talk (in private) with the Undertow lead at Red Hat, and she told me that the project was not being phased out at all, and that they will continue to support it in the long term.
• I definitely have issues in production with everything that is not Tomcat, as this is what most companies and cloud providers are expecting and monitoring (including Azure, where I work)
• I've never thought of "policy 1", @pascalgrimaud this is a very good argument. Undertow complexifies our configuration a bit, so we shouldn't have it without a good reason.
• This leaves us with performance, because I believe this is the heart of the discussion: I can easily do performance tests if I get the time. I will try to prioritize this, as I believe this would be interesting to a lot of people.

[cbornet]

Just for information, for the reactive stack we use Netty which is Webflux's default.

[jdubois]

I've just created this repository to do some tests:

https://github.com/jdubois/jhipster-benchmarks

It contains 3 applications: Tomcat, Undertow and Webflux (= Netty), so that's pretty self-explanatory.

Those applications are very light: no database, no cache, etc... So this makes the application server start-up time even more important.

I have configured them with Gatling, so we'll be able to do some "real" performance tests, but at the moment I've only done start-up performance tests. But those are what everybody looks at today, so that's a good start.

Here is my configuration for those tests:

• The default VM that we have on Azure -> Standard D2s v3 (2 vcpus, 8 GiB memory)
• The default JVM that comes with AdoptOpenJDK -> openjdk version "11.0.6" 2020-01-14

So nothing strange/fancy/optimized here, just the defaults.

I'm running those applications using "./mvnw -Pprod,tls":

• We benefit from being in an exploded hierarchy, which gives better start-up performance: this is what we would have with Docker or when running from an IDE, so I believe this is the most classical use-case
• I'm using our "tls" profile to have SSL+HTTP2 together, as I believe this is the goal for JHipster 7

Here are the results of 10 rounds of each:

	Undertow	Tomcat	Webflux
1	4,879	5,237	4,285
2	4,847	5,125	4,225
3	4,889	5,103	4,221
4	5,013	5,129	4,232
5	4,84	5,134	4,271
6	5,007	5,141	4,191
7	4,868	5,214	4,147
8	4,826	5,032	4,251
9	4,856	5,069	4,274
10	4,908	5,078	4,128
Mean	4,8933	5,1262	4,2225
Difference		4,76%	-13,71%

So Tomcat is less performant than Undertow, but that difference is very small. It's less than 5%, and with a real world app (with a database, cache, etc), it should be less than 2-3%.

[patrickjp93]

@jdubois Thank you for the info. I figured I'd repeat the same startup tests using the Open J9 build of 11.0.6 from AdoptOpenJDK, because for some users, memory constraints in multi-tenanted environments are an important consideration.

I'm on the latest baremetal ClearOS and using a ClearOS Docker Image on a NUC8I7BEH, so 2400MHz RAM, 8vCPUs, a Samsung 970 EVO.

	Undertow	Tomcat	Webflux
1	3.877	4.211	3.301
2	3.901	4.252	3.407
3	3.872	4.238	3.409
4	3.868	4.22	3.358
5	3.87	4.301	3.311
6	3.871	4.199	3.411
7	3.88	4.231	3.376
8	3.877	4.24	3.381
9	3.874	4.233	3.365
10	3.869	4.219	3.402
Mean	3.8759	4.2344	3.3721
Difference		9.249%	-12.998%

Not sure if OpenJ9 is better optimized for multi-core startup or if Webflux is (or both), but Webflux on my NUC has startup times 20% faster than Tomcat on OpenJ9. Webflux and Tomcat did have much more variability though at 0.11 seconds difference from tail to tail vs. 0.04 for Undertow.

[patrickjp93]

Repeated the same with the latest Java 14 build, pretty much the same performance deltas, but all startup time averages are down 0.22-0.26 seconds.

[patrickjp93]

And repeated with the 11.0.6 Hotspot (classic) JVM.

	Undertow	Tomcat	Webflux
Mean	3.36	3.801	3.04
Difference		13.125%	-9.524%

I know Tomcat is a synchronous server; but since startup should be file I/O bound, either Webflux is NVMe aware in ways Tomcat is not yet (which I doubt), or there's something in the multithreaded optimization. Or Clear Linux COULD be playing favorites, but I'd like to think this isn't the case.

[jdubois]

I just did full performance tests, and created a blog with all the details: https://dev.to/azure/spring-boot-performance-benchmarks-with-tomcat-undertow-and-webflux-4d8k

End result: Undertow seems to be the winner :-)

[atomfrede]

Very interesting results. Didn't expect that.

[jdubois]

Neither did I!!!! That's also why I'm totally open to comments.

[gmarziou]

Undertow 3 will be built on top of Netty and will be an intermediate release until version 4 that will be the real one. SO maybe this is something to take into account.

[patrickjp93]

@gmarziou Long live complexity! Well if that's going to be the case, SHOULD JHipster abandon Undertow if its underlying implementation is going to be highly supported by Netty? The 3.x releases may lack "perfect" stability going by the documentation, but it also sounds like the pain period will be short (maybe 4-6 months?).

Do we do all of the hard work to move to Tomcat (which yes has ready-made PCI-DSS network and application hardening guides--a benefit for some industries) if Undertow will be supported long-term and has the existing integration?

Is it possible to clean up/streamline the Undertow integration to alleviate some pain points?

Is there a potential benefit of Undertow using Netty which lets us unify additional parts of the Webflux deployment with the Undertow monolith deployments?

[armdev]

Hi all, I guess we need faster spring which is not depend from server type.

[cbornet]

Would it be possible to introduce profiles to choose the server at runtime ?

[pascalgrimaud]

@cbornet : you make me remember old version of JHipster (2.21.0 for example), we had multiple profile, specially fast profile with Undertow. See https://github.com/jhipster/generator-jhipster/blob/v2.21.0/app/templates/_pom.xml#L815-L840 and default profile with Tomcat

[armdev]

Problem is that tomcat and undertow mapped with Servlet specification, an webflux with reactive specification. Comfortable is having netty with servlet /web or webflux as a library , in other side not all microservices can be reactive

[cbornet]

@cbornet : you make me remember old version of JHipster (2.21.0 for example), we had multiple profile, specially fast profile with Undertow. See https://github.com/jhipster/generator-jhipster/blob/v2.21.0/app/templates/_pom.xml#L815-L840 and default profile with Tomcat

Yes I remember now. And I also think we removed it because we said that we had too much profiles and it complexified the pom.xml :smile:

[armdev]

I have pushed project with live benchmarks, may be it will useful for you https://github.com/armdev/game-of-thrones

[atomfrede]

@jdubois How much time would it take to do the same benchmarks again, but this time comparing hotspot and j9, especially with regards to startup and memory consumption?

[jdubois]

@atomfrede I need a few hours, probably between 2 and 4. But then we can't really depend on a JVM, we can't tell people to use one or the other.

[atomfrede]

That's true but at least for our docker builds we can do that (right now we define hotspot for example).

[mraible]

Can we close this? If the current solution is "good enough", do we need to change it? Please re-open this issue if you feel that we should.