Quarkus websocket performance degradation compared to netty

[agrikhno]

Describe the bug

I was wondering if there was a difference in performance between Quarkus websocket implementation and Netty and to find out I did load testing. I wrote a test websocket servers in quarkus and netty as well as websocket clients in quarkus and netty. Source code links are below.

The load testing context is as follows: there are 10 rooms on the server. Each room is connected to 5 clients every 15 seconds. After connecting, each client sends a random string ~2.5K characters long to the server every random(500, 1000) milliseconds. The server, after receiving a message from the client, sends it to all connected clients in the room.

VisualVM used to obtain metrics Netty version 4.1.90.Final Quarkus version 3.4.2

Below are the metrics for the quarkus websocket client and netty websocket client:

Netty websocket client metrics:

Quarkus websocket client metrics:

Below are the metrics for the quarkus websocket server and netty websocket server:

Netty websocket server metrics:

Quarkus websocket server metrics:

Expected behavior

Quarkus performance is expected to be as good as netty since quarkus websocket implementation library uses netty under hood

Actual behavior

1. Quarkus websocket client implementation consumes more than 2.5 times more CPU resources and heap memory compared to Netty
2. Quarkus websocket server implementation cannot utilize more than 20% of CPU and at a certain load level, a memory leak occurs

How to Reproduce?

Source code of tests available:

1. Jakarta websocket clients https://github.com/agrikhno/jakarta-ws-client
2. Netty websocket clients https://github.com/agrikhno/netty-ws-client
3. Jakarta websocket server https://github.com/agrikhno/jakarta-ws-server
4. Netty websocket clients https://github.com/agrikhno/netty-ws-server

Output of uname -a or ver

Darwin MacBook-Pro.local 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000 arm64

Output of java -version

Java HotSpot(TM) 64-Bit Server VM (build 17.0.5+9-LTS-191, mixed mode, sharing)

GraalVM version (if different from Java)

No response

Quarkus version or git rev

3.4.2

Build tool (ie. output of mvnw --version or gradlew --version)

Gradle 8.1.1

Additional information

No response

[mkouba]

@agrikhno It would be nice if you could include the experimental quarkus-websocket-next in the server part of your benchmarks.

[agrikhno]

@agrikhno It would be nice if you could include the experimental quarkus-websocket-next in the server part of your benchmarks.

@mkouba I did it.

Test environment: the same. Test scenario: the same.

I did two benchmarks - using blocking and non blocking methods (See metrics below.):

1. When using non blocking Uni return type and non blocking sendText(M message), I got results close to the results of previous testing io.quarkus:quarkus-websockets library in the server part, namely:

   • low cpu utilization (up to 30-35%)
   • hight Direct Byte Buffer pool memory consumption (up to ~100Mb)
   • smaller number of connected clients compared to other tests.

2. When using blocking void return type and blocking sendTextAndAwait(M message), I got better result:

   • better cpu utilization (up to 60-65%)
   • low Direct Byte Buffer pool memory consumption (up to 650Kb)
   • more connected and served clients

Non blocking mode:

Blocking mode:

[mkouba]

@agrikhno It would be nice if you could include the experimental quarkus-websocket-next in the server part of your benchmarks.

@mkouba I did it.

Test environment: the same. Test scenario: the same.

Thanks @agrikhno. We will take a look.

Could you share the source for the quarkus-websocket-next server app? Also what version of Quarkus did you use? We did some perf improvements recently.

[agrikhno]

@mkouba Version of Quarkus is 3.10.1, source code here https://github.com/agrikhno/quarkus-ws-server-next .

[mkouba]

@mkouba Version of Quarkus is 3.10.1, source code here https://github.com/agrikhno/quarkus-ws-server-next .

Ok, thanks!

[cescoffier]

@mkouba There is an improvement coming in Vert.x (should be release soon). We might need to revisit this once merged.