search

hanachin / with_refinements

MIT License
2 stars 0 forks source link

Does it have any benchmark? #1

Open st0012 opened 6 years ago

st0012 commented 6 years ago

I think the usage of this gem is far more straightforward than the original refinement. But I guess all magics come with a price. So I'm wondering if there's any benchmark to compare this gem with normal refinements?

hanachin commented 6 years ago

It doesn't have benchmark yet, sorry.

with_refinements does a lot of jobs. When I call with_refinements it actually does these steps, I think it is slow.

  1. create a disposal class and a disposal modules to get binding with refinements.
  2. copy the block's local variables
  3. read the entire ruby script file to get source code of the block
  4. copy changed local variables back at the end. (Not implemented yet)

Fortunately, there are many gems that using refinements, that used in the real world applications. I guess we can use the gem's test suite as benchmark by modifying these gem's code to with_refinements version.

hanachin commented 6 years ago

I added a microbenchmark. https://github.com/hanachin/with_refinements/blob/v0.5.0/examples/b.rb

I added a cache for refined proc to improve performance, but it is way more slower than plain refinements.

The result in my machine:

ruby examples/b.rb
Warming up --------------------------------------
           plain monkey patch     4.632M i/s -      4.653M times in 1.004581s (215.90ns/i, 319clocks/i)
            plain refinements     4.756M i/s -      4.798M times in 1.008800s (210.24ns/i, 578clocks/i)
             with_refinements    54.199k i/s -     55.121k times in 1.017005s (18.45μs/i)
       with_refinements_light    89.693k i/s -     91.674k times in 1.022092s (11.15μs/i)
      SinRefinements.refining    440.305 i/s -     473.000 times in 1.074256s (2.27ms/i)
SinRefinements.light_refining    73.796k i/s -     74.712k times in 1.012414s (13.55μs/i)
Calculating -------------------------------------
           plain monkey patch     6.400M i/s -     13.895M times in 2.171215s (156.25ns/i, 437clocks/i)
            plain refinements     6.308M i/s -     14.269M times in 2.262041s (158.53ns/i, 375clocks/i)
             with_refinements    61.366k i/s -    162.597k times in 2.649643s (16.30μs/i)
       with_refinements_light    94.589k i/s -    269.077k times in 2.844710s (10.57μs/i)
      SinRefinements.refining    439.103 i/s -      1.320k times in 3.006132s (2.28ms/i)
SinRefinements.light_refining    73.986k i/s -    221.387k times in 2.992269s (13.52μs/i)

Comparison:
           plain monkey patch:   6399801.8 i/s
            plain refinements:   6308115.9 i/s - 1.01x  slower
       with_refinements_light:     94588.5 i/s - 67.66x  slower
SinRefinements.light_refining:     73986.3 i/s - 86.50x  slower
             with_refinements:     61365.6 i/s - 104.29x  slower
      SinRefinements.refining:       439.1 i/s - 14574.73x  slower

FYI: there is the gem for same purpose as with_refinements gem. https://speakerdeck.com/joker1007/true-refinements-number-tqrk12 https://github.com/joker1007/sin_refinements