Some help - Githubissues

Doumor commented 3 years ago

I am thinking of transferring my current project to this library. I can make examples in return, but I need many things to start working in this lib. I'm not sure I can help in terms of code myself, since I've never written anything like this before.

I will be able to start by July 10th. I can also write performance tests and self-tests to check the functionality of some functions.

My current project is proprietary, if that matters.

Doumor commented 3 years ago

@endel, I immediately suggest that you create a folder of examples and create examples under a specific name and language in that folder. Your current example should be named somehow and placed in that folder. I don't like TypeScript and want to write my examples in JavaScript, as well as tests.

endel commented 3 years ago

Hi @Doumor, thanks for your willingness to use & contribute to this project.

I can port the examples to plain JavaScript for your reference. The test cases are all going to continue being written in TypeScript, though. TypeScript makes the project much easier to maintain, iterate, and refactor without breaking things.

Hope to see you around! Cheers!

Doumor commented 3 years ago

@endel, I mean my examples. I'll just write them in JavaScript. You can leave yours as they are. We'll talk after I add the performance tests. I'll need to update the readme.md to add the Performance header, and the test results themselves. I'll attach my performance test results here.

I'm just thinking about how to make nice performance graphs. Any ideas?

Doumor commented 3 years ago

@endel, I did the first test. I used autocannon for testing.

Express

Running 5s test @ http://localhost:3000
1000 connections

┌─────────┬────────┬────────┬────────┬────────┬───────────┬──────────┬────────┐
│ Stat    │ 2.5%   │ 50%    │ 97.5%  │ 99%    │ Avg       │ Stdev    │ Max    │
├─────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼────────┤
│ Latency │ 128 ms │ 136 ms │ 347 ms │ 393 ms │ 147.28 ms │ 45.48 ms │ 420 ms │
└─────────┴────────┴────────┴────────┴────────┴───────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%  │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 4511    │ 4511    │ 7343    │ 7515   │ 6780.4  │ 1146.77 │ 4511    │
├───────────┼─────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 1.08 MB │ 1.08 MB │ 1.76 MB │ 1.8 MB │ 1.62 MB │ 274 kB  │ 1.08 MB │
└───────────┴─────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

34k requests in 5.24s, 8.1 MB read

uWebSockets-express

Running 5s test @ http://localhost:3000
1000 connections

┌─────────┬───────┬───────┬───────┬───────┬──────────┬──────────┬────────┐
│ Stat    │ 2.5%  │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev    │ Max    │
├─────────┼───────┼───────┼───────┼───────┼──────────┼──────────┼────────┤
│ Latency │ 30 ms │ 32 ms │ 53 ms │ 70 ms │ 35.39 ms │ 25.16 ms │ 355 ms │
└─────────┴───────┴───────┴───────┴───────┴──────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 20671   │ 20671   │ 30015   │ 31967   │ 28689.6 │ 4182.55 │ 20666   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 1.92 MB │ 1.92 MB │ 2.79 MB │ 2.97 MB │ 2.67 MB │ 389 kB  │ 1.92 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

143k requests in 5.27s, 13.3 MB read

The performance has increased by 4 times compared to the Express. The latency has decreased.

Do you think it makes sense to do a multi-threaded hello world test?

I want to do another test with a full CSS page, but I'm not sure yet that autocannon can request attachments after receiving a response.

I'll definitely do a test with a .html document as the response and an ejs handler.

What tests would you suggest yourself?

Pull request will be when I finish all the tests and design.

Doumor commented 3 years ago

@endel, I did the second test.

Express

Running 30s test @ http://localhost:3000
1000 connections

┌─────────┬────────┬────────┬────────┬────────┬───────────┬──────────┬─────────┐
│ Stat    │ 2.5%   │ 50%    │ 97.5%  │ 99%    │ Avg       │ Stdev    │ Max     │
├─────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼─────────┤
│ Latency │ 259 ms │ 276 ms │ 562 ms │ 599 ms │ 298.67 ms │ 77.45 ms │ 2256 ms │
└─────────┴────────┴────────┴────────┴────────┴───────────┴──────────┴─────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev  │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Req/Sec   │ 1405    │ 1405    │ 3571    │ 3779    │ 3336.84 │ 602.83 │ 1405    │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Bytes/Sec │ 2.25 MB │ 2.25 MB │ 5.73 MB │ 6.06 MB │ 5.35 MB │ 967 kB │ 2.25 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴────────┴─────────┘
┌──────┬────────┐
│ Code │ Count  │
├──────┼────────┤
│ 200  │ 100091 │
└──────┴────────┘

Req/Bytes counts sampled once per second.

100k requests in 30.2s, 161 MB read

uWebSockets-express

Running 30s test @ http://localhost:3000
1000 connections

┌─────────┬────────┬────────┬────────┬────────┬───────────┬──────────┬────────┐
│ Stat    │ 2.5%   │ 50%    │ 97.5%  │ 99%    │ Avg       │ Stdev    │ Max    │
├─────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼────────┤
│ Latency │ 103 ms │ 106 ms │ 114 ms │ 117 ms │ 107.21 ms │ 10.68 ms │ 298 ms │
└─────────┴────────┴────────┴────────┴────────┴───────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev  │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Req/Sec   │ 8191    │ 8191    │ 9351    │ 9503    │ 9291.54 │ 234.84 │ 8191    │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Bytes/Sec │ 11.9 MB │ 11.9 MB │ 13.6 MB │ 13.8 MB │ 13.5 MB │ 341 kB │ 11.9 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴────────┴─────────┘
┌──────┬────────┐
│ Code │ Count  │
├──────┼────────┤
│ 200  │ 278744 │
└──────┴────────┘

Req/Bytes counts sampled once per second.

279k requests in 30.18s, 406 MB read

Fastify

Running 30s test @ http://localhost:3000
1000 connections

┌─────────┬────────┬────────┬────────┬────────┬───────────┬──────────┬────────┐
│ Stat    │ 2.5%   │ 50%    │ 97.5%  │ 99%    │ Avg       │ Stdev    │ Max    │
├─────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼────────┤
│ Latency │ 128 ms │ 133 ms │ 148 ms │ 153 ms │ 135.14 ms │ 14.53 ms │ 373 ms │
└─────────┴────────┴────────┴────────┴────────┴───────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev  │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Req/Sec   │ 5955    │ 5955    │ 7443    │ 7599    │ 7373.47 │ 301.02 │ 5952    │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼────────┼─────────┤
│ Bytes/Sec │ 9.07 MB │ 9.07 MB │ 11.3 MB │ 11.6 MB │ 11.2 MB │ 459 kB │ 9.06 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴────────┴─────────┘
┌──────┬────────┐
│ Code │ Count  │
├──────┼────────┤
│ 200  │ 221196 │
└──────┴────────┘

Req/Bytes counts sampled once per second.

221k requests in 30.18s, 337 MB read

Based on the results of these two tests, I can say that I would rather use Fastify. Judging by their tests, we got results at their level. When functions like req.locals are added, the performance may drop even more. Well, I think so, because I was told that Express is fat.

Anyway, I have two options. Use uWebSockets.js and write a lot of code myself, or just switch to Fastify. I prefer the second one. And I don't consider this library at all. It's not up to the necessary level of performance, and simply nothing in it works yet, and I need to start writing code very soon.

The first test done on Fastify

Running 5s test @ http://localhost:3000
1000 connections

┌─────────┬───────┬───────┬───────┬───────┬──────────┬──────────┬────────┐
│ Stat    │ 2.5%  │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev    │ Max    │
├─────────┼───────┼───────┼───────┼───────┼──────────┼──────────┼────────┤
│ Latency │ 23 ms │ 35 ms │ 58 ms │ 89 ms │ 37.27 ms │ 23.34 ms │ 314 ms │
└─────────┴───────┴───────┴───────┴───────┴──────────┴──────────┴────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 20751   │ 20751   │ 27951   │ 29087   │ 26824   │ 3088.25 │ 20740   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.65 MB │ 3.65 MB │ 4.92 MB │ 5.12 MB │ 4.72 MB │ 544 kB  │ 3.65 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘
┌──────┬────────┐
│ Code │ Count  │
├──────┼────────┤
│ 200  │ 134117 │
└──────┴────────┘

Req/Bytes counts sampled once per second.

134k requests in 5.19s, 23.6 MB read

One of these days I'll do what you ask in terms of tests, make a pull request with what's already there and your wish, if you can find one, and then my help is done. It goes something like this. Oh, I'll also make a nice spreadsheet, I think.

colyseus / uWebSockets-express

Some help #4