Add prism (Crystal)

[waghanza]

https://github.com/vladfaust/prism

ping @vladfaust

[vladfaust]

Oh, I personally find it useless. Each Crystal framework is ultra-fast and the micro-differences only depend on amount of features a framework has. Prism uses the same Radix tree implementation as others to resolve paths, so I don't think it makes any sense.

[waghanza]

I understand you point of view, but I disagree.

Each Crystal framework is ultra-fast and the micro-differences only depend on amount of features a framework has.

If you see the results (of some other benchmarks), we notice it a difference between frameworks, and it could be interesting to display those informations :stuck_out_tongue:

[OvermindDL1]

Crystal itself is single-core fast, but as a server it is dreadfully slow because of the way these keep getting implemented. As the Crystal devs themselves stated, the servers need to be run one per core with REUSE socket enabled. Running them as they are in this benchmark is entirely wrong, so before more are added, the existing ones need to be fixed first.

[andymans]

Crystal looks like a very interesting language. However, attracting people to it on the basis of these benchmarks is counter-productive. As soon a people try doing anything beyond a trivial 'hello world' these benchmarks collapse. As an example, JSON processing in crystal is only fractionally faster than writing the data down by hand, scanning it, and sending it to the user via email :-)

[waghanza]

The purpose of this benchmarking project is not to attract people on crystal.

The main idea is to compare tools (frameworks)

[vladfaust]

@andymans, see https://carc.in/#/r/43pg

  to json 107.79k (  9.28µs) (± 1.23%) fastest
from json 119.36k (  8.38µs) (± 2.09%) fastest

100k+ on my 0.9Ghz machine. Multiplied by 4 cores ($2.99 on Scaleway) results in 400k+ per second. It should be more than enough for the most of applications :thinking:

Attracting people to it on the basis of these benchmarks is actually a good thing, because if another language framework sucks even with hello world, it's a reason to think of migrating to an initially faster solution :+1:

BTW, I'm finishing Crystal version of http://realworld.io and it literally kills other languages :gun:

[andymans]

Excellent - very much look forward to seeing the result! The realworld is a useful app specifically because it tests > 1 facet - e.g. the router, json, plus non-infrastructure items such as jwt etc. To my mind, a good score on realworld is a far more realistic yardstick of what the language can deliver.

[OvermindDL1]

and it literally kills other languages gun

How so in what way? I'd love to see a blog on how! :-)

[vladfaust]

@OvermindDL1 I'll definitely post a Medium article on this!

[OvermindDL1]

@vladfaust Awesome, ping me with the URL when it's up? :-)

[waghanza]

@vladfaust You seems to be right. On the last test https://github.com/waghanza/which_is_the_fastest/blob/wrk/README.md#ranking-framework all crystal frameworks seems to be very close (what is now true for other languages)

[vladfaust]

So, I'm benchmarking it on my Xiaomi Mi Air 12':

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
Thread(s) per core:    2
Core(s) per socket:    2
Model name:            Intel(R) Core(TM) m3-6Y30 CPU @ 0.90GHz
CPU MHz:               500.010
CPU max MHz:           2200,0000
CPU min MHz:           400,0000

Wrk script with threads = cpu_count + 1:

printf "> Running GET /\n"
wrk -t5 -d15s -c1000 http://127.0.0.1:3000
printf "\n> Running POST /\n"
wrk -s bench/wrk/post.lua -t5 -d15s -c1000 http://127.0.0.1:3000
printf "\n> Running GET /user/42\n"
wrk -t5 -d15s -c1000 http://127.0.0.1:3000/user/42

Prism

@ v0.4.0-beta.3:

require "prism"

struct UserAction
  include Prism::Action
  include Prism::Action::Params

  params do
    type id : Int32
  end

  def call
    text(params[:id])
  end
end

router = Prism::Router.new do
  on "/", methods: %w(get post)
  get "/user/:id", UserAction
end

server = Prism::Server.new([router])
server.bind_tcp(3000, reuse_port: true)
server.listen

Results for a single server instance:

``` > Running GET / Running 15s test @ http://127.0.0.1:3000 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 22.80ms 13.37ms 451.56ms 99.31% Req/Sec 8.82k 1.13k 16.35k 89.73% 658183 requests in 15.08s, 38.92MB read Requests/sec: 43635.38 Transfer/sec: 2.58MB > Running POST / Running 15s test @ http://127.0.0.1:3000 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 26.74ms 8.84ms 237.52ms 91.17% Req/Sec 7.24k 814.26 10.44k 81.20% 540225 requests in 15.07s, 31.94MB read Requests/sec: 35852.77 Transfer/sec: 2.12MB > Running GET /user/42 Running 15s test @ http://127.0.0.1:3000/user/42 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 27.32ms 20.43ms 467.08ms 95.25% Req/Sec 7.26k 1.27k 16.68k 90.93% 542030 requests in 15.04s, 53.76MB read Requests/sec: 36039.95 Transfer/sec: 3.57MB ```

Results for 4 server instances:

``` > Running GET / Running 15s test @ http://127.0.0.1:3000 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 10.64ms 8.52ms 86.31ms 73.66% Req/Sec 18.88k 3.17k 37.65k 73.50% 1403442 requests in 15.07s, 82.98MB read Requests/sec: 93142.05 Transfer/sec: 5.51MB > Running POST / Running 15s test @ http://127.0.0.1:3000 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 13.14ms 11.58ms 160.23ms 83.03% Req/Sec 15.81k 3.13k 26.50k 67.30% 1179816 requests in 15.10s, 69.76MB read Requests/sec: 78155.70 Transfer/sec: 4.62MB > Running GET /user/42 Running 15s test @ http://127.0.0.1:3000/user/42 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 36.74ms 80.77ms 1.06s 92.19% Req/Sec 12.67k 3.12k 21.46k 74.59% 939867 requests in 15.09s, 93.22MB read Requests/sec: 62296.41 Transfer/sec: 6.18MB ```

Kemal

@ v0.24.0

require "kemal"

Kemal.config do |cfg|
  cfg.serve_static = false
  cfg.logging = false
end

get "/" do |env|
  nil
end

get "/user/:id" do |env|
  env.params.url["id"]
end

post "/user" do |env|
  nil
end

Kemal.config.env = "production"
Kemal.run do |cfg|
  cfg.server.not_nil!.bind_tcp 3000, reuse_port: true
end

Results for a single server instance:

``` > Running GET / Running 15s test @ http://127.0.0.1:3000 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 27.34ms 16.93ms 467.08ms 96.37% Req/Sec 7.21k 1.22k 12.47k 90.80% 538466 requests in 15.08s, 55.46MB read Requests/sec: 35708.89 Transfer/sec: 3.68MB > Running POST / Running 15s test @ http://127.0.0.1:3000 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 80.74ms 39.34ms 935.51ms 97.81% Req/Sec 2.42k 432.41 3.42k 82.53% 180781 requests in 15.05s, 4.33GB read Requests/sec: 12014.53 Transfer/sec: 294.53MB > Running GET /user/42 Running 15s test @ http://127.0.0.1:3000/user/42 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 35.61ms 48.60ms 971.49ms 98.75% Req/Sec 6.11k 1.52k 16.65k 91.67% 445184 requests in 15.05s, 46.70MB read Requests/sec: 29573.52 Transfer/sec: 3.10MB ```

Results for 4 server instances:

``` > Running GET / Running 15s test @ http://127.0.0.1:3000 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 23.08ms 57.65ms 920.91ms 94.80% Req/Sec 15.86k 3.27k 31.29k 74.01% 1179546 requests in 15.09s, 121.49MB read Requests/sec: 78157.36 Transfer/sec: 8.05MB > Running POST / Running 15s test @ http://127.0.0.1:3000 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 58.09ms 29.44ms 788.61ms 94.35% Req/Sec 3.47k 700.61 5.02k 63.34% 257917 requests in 15.08s, 6.17GB read Requests/sec: 17108.43 Transfer/sec: 419.36MB > Running GET /user/42 Running 15s test @ http://127.0.0.1:3000/user/42 5 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 30.82ms 70.33ms 770.44ms 94.50% Req/Sec 12.00k 3.25k 28.77k 73.01% 890524 requests in 15.09s, 93.42MB read Requests/sec: 59026.08 Transfer/sec: 6.19MB ```

TL;DR: Requests/sec

i stands for instance

	1i GET /	4i GET /	1i POST /	4i POST /	1i GET /user/42	4i GET/user/42
Prism	43.6k	93.1k	35.8k	78.1k	36.0k	62.2k
Kemal	35.7k	78.1k	12.0k	17.1k	29.5k	59.0k
Raze	46.2k	91.9k	21.5k	30.3k	38.1k	62.3k

---

@sdogruyol I took Kemal code from this repo, it may be outdated, can it be improved for more performance?

Edit: I took Kemal for comparison because it's the one I've had experience with, and it is "sinatra-like", just like Prism. I haven't benched other frameworks.

Edit: Added Raze.

[waghanza]

@vladfaust Thanks for update :stuck_out_tongue: So, you say that the assertion you made

Each Crystal framework is ultra-fast and the micro-differences

is not true

btw, the results here are no ready for any conclusion ;-) the code is quite update, so thanks for yours :heart:

[aichholzer]

@andymans -This is probably the best and most to-the-point comment I have ever read on a code thread. I keep giggling every time I read it... 😆

[waghanza]

@aichholzer the comment above is indeed very interesting, but the assumption that this project is trying to attract people in crystal community is wrong ;-)

[vladfaust]

@waghanza we should find a way to test multiple instances of Crystal servers.

[waghanza]

@vladfaust what do you mean by multiple instances ?

[stakach]

@vladfaust I did: https://github.com/the-benchmarker/web-frameworks/pull/333#issuecomment-417858485

[waghanza]

ah ok, using all available cores on each machines ... has to be implemented on each frameworks

if you have time to PR, I'll be glad to learn :heart: or I'll implement later :stuck_out_tongue: