search

mroth / phoenix-showdown

:horse_racing: benchmark Sinatra-like web frameworks
705 stars 70 forks source link

Add D with Vibe.D and vagrantfile #29

Open yamadapc opened 8 years ago

yamadapc commented 8 years ago

Adds vibe.d D implementation. To get the best performance results, it should be compiled using ldc2 or gdc on Linux.

On my Macbook Pro i7 machine, running inside a VM, the GDC compiled version has roughly 2x throughput and almost 10x lower latency than the Golang version, when running with a single CPU.

When the number of CPUs increases (I tested with 4), the results change. The Golang version seems to be better at taking advantage of all the cores. However, when scaling wrk to use more threads and more connections, the D version starts getting faster; on my tests, up to 33% faster with 1/3x lower latency.

In all tests, the maximum latency (probably caused by a GC pause) is much lower in D, staying bellow 30ms even with 100 connections and 8 threads from wrk.

The results on Ubuntu were:

Framework Throughput (req/s) Latency (ms) Max Latency (ms)
Gin (10c 2t) 33338.40 0.94 32.26
Gin (100c 8t) 29933.99 9.49 254.92
Vibe.d (GDC - 10c 2t) 26062.57 0.38 6.85
Vibe.d (GDC - 100c 8t) 38153.09 2.48 20.60
Vibe.d (LDC - 10c 2t) 24922.42 0.43 8.60
Vibe.d (LDC - 100c 8t) 35869.74 2.85 31.98

See the commit message for more.

Also adds a vagrantfile to compile the Go and Vibe.d versions on a VirtualBox environment with 2048MB of RAM and 4 CPUs. It also includes wrk on the VM for benchmarking.

To compile D, it installs GDC, which is the GCC frontend for D, and LDC, which is the LLVM frontend for D.

jadbox commented 8 years ago

I think the performance difference between the two could lie in the templating engine and string handling. Most of my tests with pure network handling seem to perform higher with Go, but I lack the experience of tuning D for these use-cases.

yamadapc commented 8 years ago

Yeah... It's a very close call, not indicative of any real performance metric. I wouldn't doubt Go has a more optimised networking stack than Vibe.D. It's just fun seeing one get so close to the other.

Usually compiling with LDC or GDC instead of DMD gives a significant performance boost to D programs. As far as more strict benchmarking goes, I think techempower does the job.

Vibe.D templates are converted to native code at compile time, as far as I know, so I wouldn't doubt that indeed D had an edge on template rendering.

The size of responses is also slightly different, though I think Go is actually sending less data than Vibe.D here. The commit message has the full wrk output for those types of comparisons.