Closed cmcinnes-mdsol closed 5 years ago
The results from the benchmark do not indicate any slow down with the use of sign
and verify
:
C02V716AHTD8:mauth-client-ruby(feature/use-rsa-sign-and-verify-for-v2) cmcinnes$ be rake benchmark
A short request has a body of 60 chars.
An average request has a body of 60,000 chars.
A huge request has a body of 6,000,000 chars.
A qs request has a body of 60 chars and a query string with two k/v pairs.
Warming up --------------------------------------
v1-sign-short 110.000 i/100ms
v2-sign-short 110.000 i/100ms
both-sign-short 57.000 i/100ms
v2-sign-qs 112.000 i/100ms
both-sign-qs 57.000 i/100ms
v1-sign-average 99.000 i/100ms
v2-sign-average 102.000 i/100ms
both-sign-average 51.000 i/100ms
v1-sign-huge 6.000 i/100ms
v2-sign-huge 7.000 i/100ms
both-sign-huge 3.000 i/100ms
Calculating -------------------------------------
v1-sign-short 1.157k (± 2.0%) i/s - 5.830k in 5.040522s
v2-sign-short 1.147k (± 2.2%) i/s - 5.830k in 5.085782s
both-sign-short 547.262 (± 3.7%) i/s - 2.736k in 5.005867s
v2-sign-qs 1.067k (± 3.5%) i/s - 5.376k in 5.044523s
both-sign-qs 549.202 (± 4.2%) i/s - 2.793k in 5.094753s
v1-sign-average 1.001k (± 3.0%) i/s - 5.049k in 5.047007s
v2-sign-average 1.016k (± 2.6%) i/s - 5.100k in 5.021087s
both-sign-average 489.535 (± 2.0%) i/s - 2.448k in 5.002828s
v1-sign-huge 63.174 (± 3.2%) i/s - 318.000 in 5.041147s
v2-sign-huge 73.135 (± 2.7%) i/s - 371.000 in 5.077876s
both-sign-huge 34.653 (± 2.9%) i/s - 174.000 in 5.025533s
Comparison:
v1-sign-short: 1157.1 i/s
v2-sign-short: 1146.9 i/s - same-ish: difference falls within error
v2-sign-qs: 1067.0 i/s - 1.08x slower
v2-sign-average: 1016.4 i/s - 1.14x slower
v1-sign-average: 1001.3 i/s - 1.16x slower
both-sign-qs: 549.2 i/s - 2.11x slower
both-sign-short: 547.3 i/s - 2.11x slower
both-sign-average: 489.5 i/s - 2.36x slower
v2-sign-huge: 73.1 i/s - 15.82x slower
v1-sign-huge: 63.2 i/s - 18.32x slower
both-sign-huge: 34.7 i/s - 33.39x slower
i/s means the number of signatures of a message per second.
Warming up --------------------------------------
v1-authenticate-short
1.019k i/100ms
v2-authenticate-short
1.052k i/100ms
v2-authenticate-qs 1.031k i/100ms
v1-authenticate-average
527.000 i/100ms
v2-authenticate-average
521.000 i/100ms
v1-authenticate-huge 7.000 i/100ms
v2-authenticate-huge 7.000 i/100ms
Calculating -------------------------------------
v1-authenticate-short
10.601k (± 5.5%) i/s - 52.988k in 5.014769s
v2-authenticate-short
11.033k (± 2.8%) i/s - 55.756k in 5.057369s
v2-authenticate-qs 10.995k (± 2.1%) i/s - 55.674k in 5.065813s
v1-authenticate-average
5.497k (± 3.3%) i/s - 27.931k in 5.087195s
v2-authenticate-average
5.367k (± 5.1%) i/s - 27.092k in 5.061478s
v1-authenticate-huge 74.951 (± 2.7%) i/s - 378.000 in 5.048807s
v2-authenticate-huge 74.568 (± 4.0%) i/s - 378.000 in 5.077331s
Comparison:
v2-authenticate-short: 11033.4 i/s
v2-authenticate-qs: 10994.9 i/s - same-ish: difference falls within error
v1-authenticate-short: 10600.6 i/s - same-ish: difference falls within error
v1-authenticate-average: 5496.7 i/s - 2.01x slower
v2-authenticate-average: 5367.0 i/s - 2.06x slower
v1-authenticate-huge: 75.0 i/s - 147.21x slower
v2-authenticate-huge: 74.6 i/s - 147.96x slower
i/s means the number of authentication checks of signatures per second.
I was hoping for a speedup :/
Other of my crazy tools https://rbspy.github.io/
Probably you can use it with rake benchmark
(or a modified version of that) and try to find slow methods
Benchmark results: After current state of V2 work:
cmcinnes$ be rake benchmark
A short request has a body of 60 chars.
An average request has a body of 60,000 chars.
A huge request has a body of 6,000,000 chars.
A qs request has a body of 60 chars and a query string with two k/v pairs.
Warming up --------------------------------------
v1-sign-short 107.000 i/100ms
v2-sign-short 108.000 i/100ms
both-sign-short 54.000 i/100ms
v2-sign-qs 102.000 i/100ms
both-sign-qs 48.000 i/100ms
v1-sign-average 87.000 i/100ms
v2-sign-average 89.000 i/100ms
both-sign-average 44.000 i/100ms
v1-sign-huge 6.000 i/100ms
v2-sign-huge 7.000 i/100ms
both-sign-huge 3.000 i/100ms
Calculating -------------------------------------
v1-sign-short 1.099k (± 3.2%) i/s - 5.564k in 5.068771s
v2-sign-short 1.076k (± 4.2%) i/s - 5.400k in 5.029293s
both-sign-short 554.546 (± 6.0%) i/s - 2.808k in 5.084830s
v2-sign-qs 1.086k (± 4.2%) i/s - 5.508k in 5.082599s
both-sign-qs 564.385 (± 3.7%) i/s - 2.832k in 5.025071s
v1-sign-average 953.878 (± 4.4%) i/s - 4.785k in 5.026740s
v2-sign-average 977.510 (± 4.0%) i/s - 4.895k in 5.015442s
both-sign-average 470.859 (± 9.8%) i/s - 2.332k in 5.009478s
v1-sign-huge 62.653 (± 3.2%) i/s - 318.000 in 5.083412s
v2-sign-huge 69.126 (± 4.3%) i/s - 350.000 in 5.072432s
both-sign-huge 34.023 (± 2.9%) i/s - 171.000 in 5.029113s
Comparison:
v1-sign-short: 1098.9 i/s
v2-sign-qs: 1085.6 i/s - same-ish: difference falls within error
v2-sign-short: 1075.6 i/s - same-ish: difference falls within error
v2-sign-average: 977.5 i/s - 1.12x slower
v1-sign-average: 953.9 i/s - 1.15x slower
both-sign-qs: 564.4 i/s - 1.95x slower
both-sign-short: 554.5 i/s - 1.98x slower
both-sign-average: 470.9 i/s - 2.33x slower
v2-sign-huge: 69.1 i/s - 15.90x slower
v1-sign-huge: 62.7 i/s - 17.54x slower
both-sign-huge: 34.0 i/s - 32.30x slower
i/s means the number of signatures of a message per second.
Warming up --------------------------------------
v1-authenticate-short
1.064k i/100ms
v2-authenticate-short
1.071k i/100ms
v2-authenticate-qs 1.065k i/100ms
v1-authenticate-average
522.000 i/100ms
v2-authenticate-average
524.000 i/100ms
v1-authenticate-huge 7.000 i/100ms
v2-authenticate-huge 7.000 i/100ms
Calculating -------------------------------------
v1-authenticate-short
10.754k (± 2.1%) i/s - 54.264k in 5.048179s
v2-authenticate-short
10.863k (± 1.6%) i/s - 54.621k in 5.029557s
v2-authenticate-qs 10.768k (± 1.5%) i/s - 54.315k in 5.045477s
v1-authenticate-average
5.294k (± 1.7%) i/s - 26.622k in 5.030435s
v2-authenticate-average
5.247k (± 2.6%) i/s - 26.724k in 5.096950s
v1-authenticate-huge 78.661 (± 2.5%) i/s - 399.000 in 5.074456s
v2-authenticate-huge 78.729 (± 2.5%) i/s - 399.000 in 5.070270s
Comparison:
v2-authenticate-short: 10862.7 i/s
v2-authenticate-qs: 10767.6 i/s - same-ish: difference falls within error
v1-authenticate-short: 10754.1 i/s - same-ish: difference falls within error
v1-authenticate-average: 5293.8 i/s - 2.05x slower
v2-authenticate-average: 5246.7 i/s - 2.07x slower
v2-authenticate-huge: 78.7 i/s - 137.98x slower
v1-authenticate-huge: 78.7 i/s - 138.10x slower
i/s means the number of authentication checks of signatures per second
Pre V2 (commit c732342903137da2eafffc803ade876ca159d68b):
cmcinnes$ be rake benchmark
DEPRECATION: Using `stub` from rspec-mocks' old `:should` syntax without explicitly enabling the syntax is deprecated. Use the new `:expect` syntax or explicitly enable `:should` instead. Called from /Users/cmcinnes/workspace/mauth-client-ruby/Rakefile:39:in `block in <top (required)>'.
Warming up --------------------------------------
sign short 103.000 i/100ms
sign average 91.000 i/100ms
sign huge 6.000 i/100ms
Calculating -------------------------------------
sign short 1.088k (± 1.9%) i/s - 5.459k in 5.019675s
sign average 940.917 (± 1.4%) i/s - 4.732k in 5.030035s
sign huge 64.361 (± 1.6%) i/s - 324.000 in 5.035538s
Comparison:
sign short: 1087.9 i/s
sign average: 940.9 i/s - 1.16x slower
sign huge: 64.4 i/s - 16.90x slower
i/s means the number of signatures of a message per second.
Warming up --------------------------------------
authenticate short 1.074k i/100ms
authenticate average 521.000 i/100ms
authenticate huge 7.000 i/100ms
Calculating -------------------------------------
authenticate short 10.533k (± 2.6%) i/s - 52.626k in 5.000022s
authenticate average 5.250k (± 2.5%) i/s - 26.571k in 5.064685s
authenticate huge 77.470 (± 3.9%) i/s - 392.000 in 5.067860s
Comparison:
authenticate short: 10532.6 i/s
authenticate average: 5249.9 i/s - 2.01x slower
authenticate huge: 77.5 i/s - 135.96x slower
i/s means the number of authentication checks of signatures per second.
@mdsol/team-16 JIRA
This PR updates the mAuth V2 protocol to use the RSA
sign
andverify
standard methods for digital signature schemes (see docs for these methods here). This is in response to concerns raised about the current use of the non-standardprivate_encrypt
in this issue: https://github.com/mdsol/mauth/issues/59.