Closed ccnlui closed 2 years ago
I did some more testing on google cloud.
My theory is that babl as a library consumes more resources than other library like java-websocket. As a result, it reaches the system limit faster as payload size increases.
Hi @ccnlui Without knowing your test setup, and the machines that you're running on, it's almost impossible to say what the cause is. Babl certainly does use more system resources than other websocket libraries, but that is why it is much faster.
As with your previous issue in the benchmark repository, you need to make sure that the systems you are running tests on have adequate resources to execute the program.
Even using simple tools such as top
should tell you whether you are maxing out the CPU on a machine. If you are, then you cannot expect to get reliable, repeatable results. Especially in the cloud, where misbehaving instances may be throttled, further degrading system performance.
That said, Babl attempts to expose useful metrics during its operation. Please refer to the Monitoring section of the documentation:
https://babl.ws/monitoring.html
All classes ending in Printer
in the com.aitusoftware.babl.monitoring
package will dump various counters to stdout. These may give you some insight into where the problem lies.
Please also spend some time reading the Configuration section:
https://babl.ws/configuration.html
to understand what your configuration file is actually doing.
Thank you @epickrram. I'm fairly certain I'm getting the above results because the system was running out of resources (local machine, not very powerful cloud instances, ...). I tried again on some dedicated, more powerful cloud instances, and I was able to measure consistent performance, ~1ms in both cases.
I'm still exploring the limit of this library, so thank you for your advice. I'll dig into the rest of the documentation.
@epickrram "Babl certainly does use more system resources than other websocket libraries, but that is why it is much faster"
What resources does babl use more than other ws libraries? CPU or memory?
I'm evaluating babl as a replacement for Undertow as the WebSocket server of a browser-based game, what do you think?
Thanks in advance
Hi @soylomass babl achieves very low latency by burning CPU time (i.e. busy-spinning poll loops). You can configure different idle-strategies to trade off CPU vs. latency. @isolgpus has built a browser-based game using babl as the back-end, so may have more useful input (I originally designed the library for trading systems, which have slightly different operating parameters).
@soylomass While our conditions may differ I can say switching to babl was probably the best single performance improvement I ever got.
Some things you may want to take into account though -
Thanks for your helpful comments! I'll give it a try
Issue: I'm trying to understand why performance drops so much when I increased broadcast payload size from 30 bytes to 100 bytes.
The babl server is very simple: it polls an external aeron subscription in the additionWork agent (applicationWork pattern), broadcasts every message that it receives to 1 topic. There is 1 websocket client listening on the topic.
Why I think its babl server? I think I'm misconfiguring babl, but I'm not sure.
I tried testing with the same websocket client, while using another websocket library like java-websocket. And I was able to measure ~1ms latency with 30 bytes and 100 bytes payload.
I also tried testing the same babl server and websocket client on two different servers on google cloud. Very similar results...
Test results below, I'd appreciate any feedback. Thank you.
Benchmark setup: 1 websocket client 15000 msg / sec broadcast every message to 1 topic
Results - payload size: 30 bytes
Results payload size: 100 bytes
babl properties: