farquet
commented
2 years ago Alternatively or in addition to Quarkus, we could investigate Micronaut. There seems to be a benchmark which could be reused in the TechEmpower benchmarks here: https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Java/micronaut
We should consider adding a workload that represents reactive serverless Java environments, such as Quarkus and friends. Some bits of thought:
A real application would also include ORM, or connect to an outside event stream, but with Quarkus the ORM layer can only be reactive with DB2, MySQL and PostgreSQL (https://quarkus.io/guides/reactive-sql-clients), which would make deployment complicated.
For inspiration, there is a guide on measuring Quarkus performance, but probably not relevant to us since it looks at startup time and memory mostly, https://quarkus.io/guides/performance-measure. Then there is a blog post, https://quarkus.io/blog/runtime-performance, which uses a very simple REST application, https://github.com/johnaohara/quarkusRestCrudDemo, which essentially does a blocking ORM access so we would need to have a database backend too (there is also an async version in the same repo but I think the choice of database backends is even more restricted then). Other people did a test article at https://micronaut.io/2020/04/07/micronaut-vs-quarkus-vs-spring-boot-performance-on-jdk-14, with code at https://github.com/graemerocher/framework-comparison-2020, this is even more simple with one singleton service, but notable is how they care about time to first response again.