Take part in TFB competition

[plokhotnyuk]

Sources: https://github.com/TechEmpower/FrameworkBenchmarks

Latest run results: https://www.techempower.com/benchmarks/#section=test&runid=a0d6e357-e8e3-4a5b-9ef0-6175ed824b20&hw=ph&test=json

[darkfrog26]

@plokhotnyuk, do you really see much value in this? I actually built the core of YouI to leverage Undertow because of those benchmarks, but a wrapper on top of one of the fastest frameworks doesn't seem a worthwhile endeavor apart from showing how much YouI slows things down. :)

[plokhotnyuk]

beside a direct value of know how to tune your server for better performance you can get new users & contributors that will be attracted by the TFB site

[darkfrog26]

@plokhotnyuk, I've been thinking more about this and I'm not opposed. Would you be interested in collaborating to build something like this out? I'm currently releasing 0.10.0 which updates how Undertow is used to be 100% non-blocking and makes HttpHandler return Future[HttpConnection] for better asynchronous behavior.

[plokhotnyuk]

Matt, please, review a PR.

Only two simplest test cases were implemented: json and plaintext. Others require an efficient async DB driver...

Also I have build a flamegraph at 100K msg/sec rate. You can download it (in a 30 days) and explore interactively in the browser. It seems that the logging system still eats a lot of CPU (~30% according the graph).

[darkfrog26]

@plokhotnyuk, nice work. I'll have to check to see why Scribe is eating so much, it shouldn't have any noticeable impact on performance. I'd be curious to see the performance difference between your JSON library and using JsonUtil as well.

[plokhotnyuk]

AFAIK JsonUtil uses circe under hood, so for a rough picture you can just look through results of benchmarks which compare both of them for different data structures and types on JDK and GraalVM. Feel free to modify sources of benchmarks by adding call of your macro and measure it's performance with different JMH profilers (gc, perfnorm, perfasm, etc.).

[darkfrog26]

@plokhotnyuk, the CI failed on that PR

[plokhotnyuk]

I have no idea what wrong is happening in the CI... According to logs the server started successfully and have no bounding to the port errors.

Can it be binding to unexpected network interface or address?

It would be great to add an explicit logging with printing of the network address and a port on both (server and test client) sides.

[darkfrog26]

You could add: https://github.com/outr/youi/blob/master/example/jvm/src/main/resources/config.json

[plokhotnyuk]

It does not help, the error message on Travis-CI is the same: https://travis-ci.org/TechEmpower/FrameworkBenchmarks/jobs/488369520

[darkfrog26]

Strange. I'm not familiar with how they do the testing, but it sounds like a configuration issue outputting the results.

[plokhotnyuk]

I have reproduced the same error using the following command: ./tfb --mode verify --test youi.

BTW, docs are here: https://github.com/TechEmpower/FrameworkBenchmarks#quick-start-guide

[plokhotnyuk]

I have fixed binding on 0.0.0.0 and the Travis CI build is green now.

All respects to @AnderEnder who helped me a lot in that!

[darkfrog26]

That's awesome! Is there any way we can see how it stacks up against the competition?

[plokhotnyuk]

Yes, after merging of that PR to the master, we will wait for the next test round on their site and then after its completion we will see not only ranking with max throughput but also results of warming up attempts with different number of connections

[plokhotnyuk]

Below are results of benchmarks for Scala http frameworks on Citrine environment.

Json:

Plaintext:

[darkfrog26]

@plokhotnyuk, awesome! This is a good starting place to do some real performance improvements. I know you've already done some performance testing. If you're interested in submitting any updates, they would be absolutely welcome.

[darkfrog26]

To be clear, I haven't done any performance tuning of YouI to this point.