search

ysbaddaden / execution_context

10 stars 2 forks source link

wrcr, a simple-minded wrk implementation in Crystal #32

Open beta-ziliani opened 2 months ago

beta-ziliani commented 2 months ago

This client addresses the issue mentioned in this comment, opening and closing connections (unlike wrk).

With CRYSTAL_WORKERS it's possible to change the number of threads. Then, these are the additional options:

Usage: wrcr [arguments]
Note: The number of threads is set by the CRYSTAL_WORKERS environment variable.
    -f NUMBER, --fibers=NUMBER       Number of total fibers. Defaults to 20
    -r NUMBER, --requests=NUMBER     Number of requests before closing the connection. Defaults to 12
    -d NUMBER, --duration=NUMBER     Duration of the test. Defaults to 00:00:30 seconds
    -u URL, --url=URL                Specifies the address of the server. Defaults to localhost:8080
    -h, --help                       Show this help

There is no attempt at balancing the fibers, and that's why the option -f specifies the total number of fibers, resorting to the (EC:MT / preview_mt) scheduler to do it.

The number of requests is used in the loop to send to the server before closing the connection. Note: Connections are not re-allocated, instead, they're re-opened.

The duration of the test is approximated: at each request we check if we reach the time, and close gracefully.

Using the server from #31 with the EC:MT context, here's a poor-man's comparison with or without EC (three runs, 2 threads, server with 4 threads).

No EC: median of 1,332,637 requests, with MAD of 35,344. EC: median of 1,341,610 requests, with MAD of 26,381.

Using the server with preview_mt:

No EC: median 1,538,338, MAD: 179,068 EC: median 1,623,682, MAD: 17,281

It's weird that this numbers are in contrast with the server's numbers when using wrk.

straight-shoota commented 2 months ago

I was wondering why we need to build our own tool for this and whether it wouldn't be easier to add this to wrk. There is even a feature request for controlling connnections: https://github.com/wg/wrk/issues/197 And it should even be possible to implement this behaviour via Lua scripting now.

However, I believe this Crystal implementation is quite valuable and could be useful for an inverse benchmark, testing Crystal's client performance.

beta-ziliani commented 2 months ago

However, I believe this Crystal implementation is quite valuable and could be useful for an inverse benchmark, testing Crystal's client performance.

Yes, this is my main motivation to do it, this is why I don't even considered other options