crystal-lang / crystal

The Crystal Programming Language
https://crystal-lang.org
Apache License 2.0
19.39k stars 1.62k forks source link

Wrong information on docs about crystals' bcrypt performance #13602

Open BlobCodes opened 1 year ago

BlobCodes commented 1 year ago

Bug Report

The following passage can be found under the docs for Crypto::Bcrypt:

This implementation of Bcrypt is currently 50% slower than pure C solutions, so keep this in mind when selecting your cost. It may be wise to test with Ruby's bcrypt gem which is a binding to OpenBSD's implementation.

However, when testing, I could not reproduce this. I think it is outdated and should be updated.

I compared the performance between crystal (stdlib) and bcrypt-ruby. These are the performance metrics from my system:

Ruby code and results ```rb require "benchmark/ips" require "bcrypt" input = "my strong password" Benchmark.ips do |x| x.report("create hash (cost 6)") { BCrypt::Password.create(input, cost: 6) } end Benchmark.ips do |x| x.report("create hash (cost 12)") { BCrypt::Password.create(input, cost: 12) } end hash = BCrypt::Password.create(input, cost: 6) Benchmark.ips do |x| x.report("verify hash (cost 6)") { hash == input } end hash = BCrypt::Password.create(input, cost: 12) Benchmark.ips do |x| x.report("verify hash (cost 12)") { hash == input } end ``` ``` Warming up -------------------------------------- create hash (cost 6) 34.000 i/100ms Calculating ------------------------------------- create hash (cost 6) 337.954 (± 0.3%) i/s - 1.700k in 5.030329s Warming up -------------------------------------- create hash (cost 12) 1.000 i/100ms Calculating ------------------------------------- create hash (cost 12) 5.535 (± 0.0%) i/s - 28.000 in 5.058732s Warming up -------------------------------------- verify hash (cost 6) 34.000 i/100ms Calculating ------------------------------------- verify hash (cost 6) 342.480 (± 0.0%) i/s - 1.734k in 5.063066s Warming up -------------------------------------- verify hash (cost 12) 1.000 i/100ms Calculating ------------------------------------- verify hash (cost 12) 5.550 (± 0.0%) i/s - 28.000 in 5.045026s ```
Crystal code and results ```cr require "crypto/bcrypt/password" require "benchmark" input = "my strong password" Benchmark.ips do |x| x.report("create hash (cost 6)") { Crypto::Bcrypt::Password.create(input, cost: 6) } end Benchmark.ips do |x| x.report("create hash (cost 12)") { Crypto::Bcrypt::Password.create(input, cost: 12) } end hash = Crypto::Bcrypt::Password.create(input, cost: 6) Benchmark.ips do |x| x.report("verify hash (cost 6)") { hash.verify(input) } end hash = Crypto::Bcrypt::Password.create(input, cost: 12) Benchmark.ips do |x| x.report("verify hash (cost 12)") { hash.verify(input) } end ``` ``` --- Using --mcpu=native --- create hash (cost 6) 344.56 ( 2.90ms) (± 0.26%) 5.46kB/op fastest create hash (cost 12) 5.40 (185.03ms) (± 0.25%) 5.1kB/op fastest verify hash (cost 6) 342.03 ( 2.92ms) (± 0.09%) 4.48kB/op fastest verify hash (cost 12) 5.41 (184.73ms) (± 0.13%) 4.4kB/op fastest --- Default --- create hash (cost 6) 322.62 ( 3.10ms) (± 0.05%) 5.46kB/op fastest create hash (cost 12) 5.11 (195.79ms) (± 0.06%) 5.53kB/op fastest verify hash (cost 6) 320.26 ( 3.12ms) (± 0.13%) 4.48kB/op fastest verify hash (cost 12) 5.06 (197.48ms) (± 0.08%) 4.4kB/op fastest ```
Type Crystal time/op Ruby time/op
create hash (cost 6) 3.10ms 2.96ms
create hash (cost 12) 195.79ms 180.67ms
verify hash (cost 6) 3.12ms 2.92ms
verify hash (cost 12) 197.48ms 180.17ms

As you can see in the table, crystal is a fair bit slower than ruby, but definitely not near 50% (if my tests aren't flawed, on my system).

I think in most cases the statement in the docs is misleading, and I have already seen people on the crystal-lang discord search for openbsd-based bcrypt libraries because of it.

straight-shoota commented 1 year ago

Yeah, this information seems to be outdated since #3880.