[C++] Add the Drogon framework

[an-tao]

This PR adds the Drogon framework.

[waghanza]

Hi @an-tao,

I have made a comparison between drogon and libevhtp (the only C++ frameworks we have here)

And results are :tada:

Language (Runtime)	Framework (Middleware)	Average	50th percentile	90th percentile	99th percentile	99.9th percentile	Standard deviation
cpp (14)	drogon (1.0)	5.88 ms	5.10 ms	10.32 ms	17.48 ms	48.45 ms	3657.00
cpp (11)	evhtp (1.2)	7.32 ms	6.94 ms	11.55 ms	19.01 ms	60.14 ms	3659.67

Language (Runtime)	Framework (Middleware)	Requests / s	Throughput
cpp (14)	drogon (1.0)	149510.00	296.09 MB
cpp (11)	evhtp (1.2)	121979.33	118.40 MB

A question however, which version of C++ drogon is in : 14 or 17

PS : Benchmark made on

OS: Linux (version: 5.1.17-300.fc30.x86_64, arch: x86_64)
CPU Cores: 4
threads: 5, connections: 1000

[waghanza]

Also @an-tao, can we disable log ?

[an-tao]

@waghanza , C++14 and C++17 are both supported and automatically detected by cmake.

[an-tao]

@waghanza, Does the test output a lot of logs?

[waghanza]

@an-tao I don't know. The reason why I ask that is because logs are not very useful is a benchmarking use case (and disabled in others frameworks)

[waghanza]

@an-tao I display the version on results, then is this right if I say that language is C++ 14 ?

[an-tao]

@waghanza C++ 14 is fine, I prefer C++ 14/17.

[an-tao]

@waghanza I'll disable log.

[an-tao]

@waghanza ,currently, only one line of logs ("start running...") is output, which is generated by the framework. I found that you set the drogon version to 1.0.0.beta2 instead of the HEAD of the master branch, with the version the log cannot be ignored. I think it doesn't matter for benchmarking.

[waghanza]

Hi @an-tao,

In deed, I prefer to stick version for each frameworks.

It adds a useful information for readers.

As for now, having 1 line of log is not very a problem, but we need to avoid it in the near future 😁

[an-tao]

@waghanza no problem! Thank you for your excellent project!

[an-tao]

@OvermindDL1 @waghanza I've modified CMakeLists.txt. Please check it, thanks!

[OvermindDL1]

Looks good. :-)

[OvermindDL1]

Tried compiling and it's needing both a jsoncpp dependency and drogon itself, not automatically acquired, looks like the docker image grabs these manually by apt'ing in jsoncpp and git clone'ing drogon (and installing it with make install, which is requiring root of all things, root should never be needed to compile an end-user application!). Why aren't this both acquired by the build system itself? Look at cpp/evhtp, it acquires all of its dependencies itself, it can be compiled straight with just cmake and a C++ compiler, nothing else needed, total of <20k worth of uncompressed files and source needed to do it.

I'm currently acquiring and installing these manually, doesn't seem like the best end-user method (generally the programs in web-frameworks here are stock normal end-user setups that would generally be created by the end user).

[an-tao]

@OvermindDL1 , I'll make them submodules of the project.

[OvermindDL1]

@an-tao Eh, docker will already handle that, I just tend to compile everything manually of any given language directly as a test, I'm not a fan of acquiring dependencies manually or installing things as root. ^.^;

It definitely shouldn't be assuming a global library installation however.

I compiled drogon inline and installed it to a local directory instead, adjusting the commands as necessary, still compiling everything.

[an-tao]

@OvermindDL1 I could use this drogon docker image

[OvermindDL1]

@an-tao I can't use a docker image on the server I test on. But it seems that this project isn't depending on drogon anyway, it seems to assume it is installed globally, which is making it really difficult to state where it is actually installed. The CMakeLists.txt file doesn't reference a drogon library at all, so it doesn't even know how to link to it. Only thing referencing a drogon binary at all are the lines:

LINK_DIRECTORIES(/usr/local/lib)
LINK_LIBRARIES(drogon trantor pthread dl)

Which absolutely should never ever be done. Never link a system-specific global directory. Always use find_package and the other primitives instead. So the cmakelists.txt file here needs to be updated to actually use a drogon cmake library, not a system installation that can't be performed on my server here (nor many others). Remember, it should be written in a way like a normal end-user program would be.

[an-tao]

@OvermindDL1 I understand, I still have to make some stuff like FindDrogon.cmake, right?

[OvermindDL1]

@OvermindDL1 I understand, I still have to make some stuff like FindDrogon.cmake, right?

Find*.cmake is only for finding non-cmake libraries, it's slower, inefficient, uses heuristics (guesses), etc... You want to use a *Config.cmake file intead, so a DrogonConfig.cmake and a TrantorConfig.cmake. These should be generated by the drogon and trantor libraries and they give full information about all build flags, requirements, file locations, everything, and you just point cmake to those files (either by installing in a well-known global or local location or by just passing it in on the command line) and it will pick it up perfectly every time.

Though in an end-user case, it would probably be a lot easier to just git submodule it in, in this case, and just add_subdirectory it. :-)

For final distribution, drogon and trantor absolutely should generate cmake config files though, but its not necessary in this specific case.

Here are some useful links: Official Docs: https://cmake.org/cmake/help/v3.0/manual/cmake-packages.7.html Official Tutorial on Config.cmake files: https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-to-create-a-ProjectConfig.cmake-file Proper cmake dependency handling: https://cgold.readthedocs.io/en/latest/tutorials/install/managing-dependencies.html This might not be needed for these projects but useful info anyway: https://cmake.org/cmake/help/latest/module/GenerateExportHeader.html

[an-tao]

@OvermindDL1 Great, but I have to spend some time to learn :-) Thank you!

BTW: Do you have a technical blog?

[OvermindDL1]

Got it manually linked together, initial tests look very nice!

It is spitting out a lot of errors on its output though:

20190730 15:08:04.914847 UTC 19040 DEBUG [handleError] [127.0.0.1:3000--127.0.0.1:35952] - SO_ERROR = 104 Connection reset by peer - TcpConnectionImpl.cc:259

Over and over endlessly. Is it supposed to be doing this? Tests are passing without error codes though:

Running 10s test @ http://localhost:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.03ms    4.45ms 109.28ms   82.19%
    Req/Sec    38.49k     5.93k   60.57k    72.97%
  1906108 requests in 10.08s, 238.13MB read
Requests/sec: 189067.42
Transfer/sec:     23.62MB

And for comparison, here's the stock example evhtp server:

Running 10s test @ http://localhost:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.96ms    3.95ms 114.35ms   78.85%
    Req/Sec    33.39k     4.55k   48.67k    70.30%
  1658968 requests in 10.08s, 101.26MB read
Requests/sec: 164517.84
Transfer/sec:     10.04MB

One thing sticks out, it looks like it's sending excess headers that it doesn't need to, it's sending:

HTTP/1.1 200 OK
Content-Length: 0
Content-Type: text/html; charset=utf-8
Server: drogon
Date: Tue, 30 Jul 2019 15:15:16 GMT

This is not necessary for these tests, and in fact are generally not wanted as you are trying to maximize connection handling. The Server and Date lines are needless and if you have an option to not send those, then you should set it. Not including the date should gain you a little bit in speed alone as it saves the current time lookup call from the kernel, that way you can get even faster. :-)

This is the minimum headers necessary for the / path as an example (and is all the headers that a lot of the things here send):

HTTP/1.1 200 OK
Content-Length: 0
Content-Type: text/plain

[OvermindDL1]

BTW: Do you have a technical blog?

Ehh, not really. I keep creating and destroying them over the years. The current one is https://blog.overminddl1.com/ but I haven't touched it in a while again and might end up wiping it, again... ^.^;

[an-tao]

@OvermindDL1 ,Date and server headers are automatically added by drogon, and currently there are no options to disable them. In fact, drogon generates a Date header every second, so it's not a big load. But removing these headers will reduce a lot of memory copies, which will definitely improve throughput:-) I will check those extra logs and then remove them. I have bookmarked your blog ^.^

[OvermindDL1]

Date and server headers are automatically added by drogon, and currently there are no options to disable them. In fact, drogon generates a Date header every second, so it's not a big load. But removing these headers will reduce a lot of memory copies, which will definitely improve throughput:-)

Ah, you definitely want options for those. Having the wrong Server header can screw up some API's. And the Date is optional in the standard as well and can be quite costly at times, in addition, some API's might not want it. The user of the library should always have the ability to override any and all headers sent, including removal, as it might be mandated that they have to do so at times to work with some broken API client (personal experience...).

And technically, removing those headers will actually lower 'throughput' but raise 'simultaneous connection handling'. Removing them will raise throughput if the packets start getting large enough to get fragmented at certain boundaries, but with such simple responses that won't happen here, just increase number of connections handled, which is always good.

[waghanza]

I could use this drogon docker image

I prefer to use a standard image, at least a image that can be used on other C++ frameworks

BTW, good job :tada: with drogon

[waghanza]

@an-tao is this PR ready for you ?

[an-tao]

@waghanza I will remove the Server header and Date header from responses and add drogon as a submodule to the benchmark in one or two days.

[waghanza]

@an-tao ok, just let me know when this PR is ready :heart:

[an-tao]

@waghanza I've added drogon as a submodule of the benchmark suite. Please check it.

BTW: I didn't use the git submodule add command because it would make drogon as a submodule of the whole project.

[waghanza]

@an-tao it's ok for me, I'm only removing any unnecessary stuff

is this PR ready ?

[an-tao]

@waghanza ，Some removed options are useful for performance. Let me modify the config.json.

[waghanza]

@an-tao ok, I let you do. But options should reflect the real-word usage, I mean drogon SHOULD NOT be tuned here for performance

[OvermindDL1]

Although extraneous headers seem fine to be removed, that is common for real world applications, especially API servers.

The current PR builds and runs very cleanly with just a git clone of drogon and a rm -r build; cmake -H. -Bbuild -DBUILD_ORM=OFF -DCMAKE_BUILD_TYPE=Release && cmake --build build --config Release now, so that's very nice! :-)

It still logs a lot of error messages, but current results as of ten minutes ago seem worse interestingly?!:

╰─➤  /home/overminddl1/tmp/wrk/wrk -t 5 -c 1000 -d 10 http://localhost:3000/
Running 10s test @ http://localhost:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    34.85ms  138.27ms   2.00s    97.00%
    Req/Sec    13.34k     1.81k   20.06k    78.00%
  663966 requests in 10.09s, 84.85MB read
  Socket errors: connect 0, read 0, write 0, timeout 3
Requests/sec:  65772.83
Transfer/sec:      8.41MB

And this holds steady with multiple runs. evhtp is currently benching at:

╰─➤  /home/overminddl1/tmp/wrk/wrk -t 5 -c 1000 -d 10 http://localhost:3000/
Running 10s test @ http://localhost:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.02ms    4.23ms  69.90ms   77.07%
    Req/Sec    32.99k     5.87k   76.49k    76.11%
  1637147 requests in 10.10s, 99.92MB read
Requests/sec: 162120.09
Transfer/sec:      9.90MB

[OvermindDL1]

There was a force push a few seconds prior, let me test with that one now, results:

╰─➤  /home/overminddl1/tmp/wrk/wrk -t 5 -c 1000 -d 10 http://localhost:3000/
Running 10s test @ http://localhost:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.02ms    6.51ms 213.46ms   92.71%
    Req/Sec    40.49k     5.17k   63.66k    74.39%
  2009166 requests in 10.10s, 256.76MB read
Requests/sec: 198942.67
Transfer/sec:     25.42MB

That's a lot better now! :-)

And no logs this time!

[an-tao]

@OvermindDL1 Try the new config.json, the 'thread_num' option is remove by @waghanza , and the default value of it is 1, setting it to 0 means the number of threads equals the number of CPU cores.

[an-tao]

@OvermindDL1 setting the "log_level" to "WARN" disables all DEBUG logs.

BTW: if you try to use wrk with HTTP pipelining, the performance of drogon would be improved several times

[an-tao]

@waghanza I think the PR is OK, thanks! @OvermindDL1 Thank your for your help!

[OvermindDL1]

Try the new config.json, the 'thread_num' option is remove by @waghanza , and the default value of it is 1, setting it to 0 means the number of threads equals the number of CPU cores.

Just did, that was the latest results.

BTW: if you try to use wrk with HTTP pipelining, the performance of drogon would be improved several times

Pipelining isn't generally useful on API endpoints, which is what these are emulating, but just for testing sake:

╰─➤  /home/overminddl1/tmp/wrk/wrk -t 5 -c 1000 -d 10 http://localhost:3000/ -s pipelining.lua
Running 10s test @ http://localhost:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.10ms    4.47ms  58.28ms   78.26%
    Req/Sec    95.04k    12.29k  136.84k    73.47%
  4694952 requests in 10.06s, 599.98MB read
Requests/sec: 466888.05
Transfer/sec:     59.66MB

Very nice indeed!

[an-tao]

@OvermindDL1 The optimization for pipeling has no adverse effect on general API requests, after all, it adds adaptability ^_^

And I am confused about the Transfer data:

evhtp:
Requests/sec: 162120.09
Transfer/sec:      9.90MB
drogon:
Requests/sec: 198942.67
Transfer/sec:     25.42MB

They seem out of proportion.

[OvermindDL1]

They seem out of proportion.

Because:

evhtp:

╰─➤  curl -D - -G http://localhost:3000/                        
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: text/plain

drogon:

╰─➤  curl -D - -G http://localhost:3000/
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: text/plain
Server: drogon/1.0.0.beta4.1006
Date: Thu, 01 Aug 2019 14:43:18 GMT

By default evhtp doesn't add any headers than what the spec says is required, anything beyond that is the users job, and right now drogon is adding a lot of needless headers.

[an-tao]

        "enable_server_header": false,
        "enable_date_header": false

These options disable the headers, please clone the v1.0.0-bate5 tag of drogon.

RUN git clone --branch v1.0.0-beta5 https://github.com/an-tao/drogon
WORKDIR $DROGON_ROOT
RUN git submodule update --init

WORKDIR $IROOT/build
RUN cmake -DCMAKE_BUILD_TYPE=release ..
RUN make
CMD ./drogon_benchmark ../config.json

[OvermindDL1]

╰─➤  /home/overminddl1/tmp/wrk/wrk -t 5 -c 1000 -d 10 http://localhost:3000/
Running 10s test @ http://localhost:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.75ms    4.26ms  52.13ms   83.48%
    Req/Sec    41.26k     5.12k   67.80k    73.23%
  2045872 requests in 10.10s, 124.87MB read
Requests/sec: 202596.41
Transfer/sec:     12.37MB

╰─➤  curl -D - -G http://localhost:3000/                                    
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: text/plain

Very nice. :-)

[OvermindDL1]

Caught my other server not being busy, so testing across a gigabit network:

drogon:

╰─➤  /home/overminddl1/tmp/wrk/wrk -t 5 -c 1000 -d 10 http://192.168.1.89:3000/
Running 10s test @ http://daggoth:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    54.97ms   69.23ms   1.82s    83.11%
    Req/Sec     6.80k   812.55     8.97k    67.20%
  338429 requests in 10.08s, 20.66MB read
Requests/sec:  33559.17
Transfer/sec:      2.05MB

evhtp:

╰─➤  /home/overminddl1/tmp/wrk/wrk -t 5 -c 1000 -d 10 http://192.168.1.89:3000/
Running 10s test @ http://daggoth:3000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    54.74ms   68.10ms 931.37ms   82.74%
    Req/Sec     6.85k   831.09    10.84k    71.40%
  341330 requests in 10.07s, 20.83MB read
Requests/sec:  33884.29
Transfer/sec:      2.07MB

It looks like drogon's throughput is mostly in better socket handling, but when that is forced over a non-local link (no fast socket setup in the kernel) it might have a touch more slowdown there instead.

[an-tao]

@OvermindDL1 , The results of drogon and evhtp are so consistent. They seem to have common bottlenecks, such as the network.

[OvermindDL1]

@OvermindDL1 , The results of drogon and evhtp are so consistent. They seem to have common bottlenecks, such as the network.

Yep, that's what I was saying, that on the local system drogon is so much faster than evhtp but on the network they are the same. ^.^

It's not because of the network itself, it's because the kernel is having to perform a full socket setup, I'm guessing drogon handles quick setups for local connections a little better than evhtp. This can actually be useful on reverse proxied setups.

[waghanza]

@an-tao Results has been updated https://github.com/the-benchmarker/web-frameworks#results

[an-tao]

@waghanza , It looks pretty good :-) Thank you for your great work with this repo!