mroth / phoenix-showdown

:horse_racing: benchmark Sinatra-like web frameworks
705 stars 70 forks source link

add gin benchmark #3

Closed hypebeast closed 9 years ago

hypebeast commented 10 years ago

Hi Matthew,

I added a Gin (https://github.com/gin-gonic/gin) based benchmarker to the project. Gin is a more Go idiomatic Sinatra style microframework for Go. I think it would be a good addition to this showdown.

On my machine I got the following results:

$ go get github.com/gin-gonic/gin
$ GOMAXPROCS=4 MARTINI_ENV=production go run server.go
[GIN-debug] GET   /:title                   --> main.func·001 (2 handlers)
[GIN-debug] Listening and serving HTTP on :3000

$ ./wrk -t4 -c100 -d30S --timeout 2000 "http://127.0.0.1:3000/showdown"
Running 30s test @ http://127.0.0.1:3000/showdown
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.83ms    3.62ms  57.75ms   97.92%
    Req/Sec    10.77k     1.91k   14.44k    82.29%
  1220584 requests in 30.00s, 2.36GB read
Requests/sec:  40686.61
Transfer/sec:     80.40MB

Compared to Martini:

Running 30s test @ http://127.0.0.1:3000/showdown
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.77ms    4.55ms  71.54ms   82.16%
    Req/Sec     2.91k   239.18     4.31k    72.46%
  342695 requests in 30.00s, 682.07MB read
Requests/sec:  11424.21
Transfer/sec:     22.74MB
mroth commented 10 years ago

I'm going to add this to the next round of benchmarks. Initial performance results are extremely promising.

Some questions/suggestions:

  1. Looking at https://github.com/gin-gonic/gin#html-rendering, is there a reason you are not using the built-in LoadHTMLTemplates() with the templates directory? (If not, we should switch to this to keep the style as similar as possible across versions.)
  2. I noticed in your logs you called it with MARTINI_ENV=production. Does Gin look for Martini's env variables for compatibility? Or is this a typo?
  3. In the documentation I see the gin.H convenience shortcut, which looks nice because it can potentially help with the ugliest part of the Martini code, which was having to create a custom struct just to pass the data along. So unless I am mistaken I believe we can do replace that context creation with something such as context := gin.H{"Title": title, "Members": members}? Or actually even use gin.H instances entirely and get rid of the custom types? (This would be much cleaner and similar to what we are doing in other languages.)
chrismccord commented 10 years ago

All of these benchmarks are tricky to to get right, but it's worth mentioning that all the solutions today do mount a static middleware that run for each request.

hypebeast commented 10 years ago

I just updated the PR.

  1. I now use the LoadHTMLGlob() instead of parsing the templates manually. The LoadHTMLTemplates() is deprecated.
  2. This was a typo. The correct call is GOMAXPROCS=4 GIN_MODE=release go run server.go.
  3. Yes, it's possible to get rid of the custom types. I updated the code accordingly. But I think the same should be possible for the Martini code, too.
mroth commented 9 years ago

FYI I am working towards integrating this in PR #8, if you wish to follow along!