fastruby / fast-ruby

:dash: Writing Fast Ruby :heart_eyes: -- Collect Common Ruby idioms.
https://github.com/fastruby/fast-ruby
5.67k stars 376 forks source link

Hash#fetch with second argument is slower than Hash#fetch with block is not FAIR #117

Closed greyblake closed 5 years ago

greyblake commented 8 years ago

Hi! First of all thanks for the nice gem! You are doing great work, making thousand of ruby programs run faster!

Now.. the issue)

Offense Hash#fetch with second argument is slower than Hash#fetch with blockis not fair.

You have this benchamrk: https://github.com/JuanitoFatas/fast-ruby/blob/master/code/hash/fetch-vs-fetch-with-block.rb

require "benchmark/ips"

HASH = { writing: :fast_ruby }
DEFAULT = "fast ruby"

Benchmark.ips do |x|
  x.report("Hash#fetch + const") { HASH.fetch(:writing, DEFAULT) }
  x.report("Hash#fetch + block") { HASH.fetch(:writing) { "fast ruby" } }
  x.report("Hash#fetch + arg")   { HASH.fetch(:writing, "fast ruby") }
  x.compare!
end

But it's assuming that key is always present. What is not really? Otherwise why one passes default value?

Here is my benchmark with hit and miss cases:

require 'benchmark'

N = 10_000_000

hash = { a: 10, b: 20, c: 30, e: 40 }

Benchmark.bm(15, "rescue/condition") do |x|
  x.report("with 2nd arg (hit) ") do
    N.times { hash.fetch(:a, false) }
  end

  x.report("with block (hit)   ") do
    N.times { hash.fetch(:a) { false } }
  end

  x.report("with 2nd arg (miss)") do
    N.times { hash.fetch(:x, false) }
  end

  x.report("with block (miss)  ") do
    N.times { hash.fetch(:x) { false } }
  end
end

Here is the output (ruby 2.1.5p273):

                      user     system      total        real
with 2nd arg (hit)   1.050000   0.000000   1.050000 (  1.041781)
with block (hit)     1.030000   0.000000   1.030000 (  1.031664)
with 2nd arg (miss)  1.010000   0.000000   1.010000 (  1.010886)
with block (miss)    1.760000   0.000000   1.760000 (  1.755389)

You see that with 2nd arg (hit) and with 2nd arg (hit) is almost the same, but diff between with 2nd arg (miss) and with block (miss) is quite big.

So, IMHO, this offense is not fair and it should be removed.

Thanks!

seanabrahams commented 6 years ago

Have to agree. Here's some output from ruby 2.3.4p301 which has the block format slower on my machine:

                      user     system      total        real
with 2nd arg (hit)   0.580000   0.000000   0.580000 (  0.578319)
with block (hit)     0.610000   0.000000   0.610000 (  0.616988)
with 2nd arg (miss)  0.630000   0.000000   0.630000 (  0.631783)
with block (miss)    0.990000   0.000000   0.990000 (  0.989013)
eclemens commented 5 years ago

The same for me using ruby-2.4.4p296

                      user     system      total        real
with 2nd arg (hit)   0.570000   0.000000   0.570000 (  0.571392)
with block (hit)     0.570000   0.000000   0.570000 (  0.569225)
with 2nd arg (miss)  0.570000   0.000000   0.570000 (  0.568199)
with block (miss)    1.030000   0.000000   1.030000 (  1.037917)
eclemens commented 5 years ago

Using benchmark-ips (2.0+) as required.

require 'benchmark/ips'

hash = { a: 10, b: 20, c: 30, e: 40 }

Benchmark.ips do |x|
  x.report("with 2nd arg (hit) ") do |n|
    n.times { hash.fetch(:a, false) }
  end

  x.report("with block (hit)   ") do |n|
    n.times { hash.fetch(:a) { false } }
  end

  x.report("with 2nd arg (miss)") do |n|
    n.times { hash.fetch(:x, false) }
  end

  x.report("with block (miss)  ") do |n|
    n.times { hash.fetch(:x) { false } }
  end

  # Compare the iterations per second of the various reports!
  x.compare!
end
Warming up --------------------------------------
 with 2nd arg (hit)    280.275k i/100ms
 with block (hit)      292.269k i/100ms
 with 2nd arg (miss)   274.662k i/100ms
 with block (miss)     255.428k i/100ms
Calculating -------------------------------------
 with 2nd arg (hit)      16.897M (± 6.1%) i/s -     84.082M in   4.998932s
 with block (hit)        16.514M (± 8.6%) i/s -     81.835M in   5.003357s
 with 2nd arg (miss)     17.672M (± 3.0%) i/s -     88.441M in   5.009506s
 with block (miss)        9.836M (± 2.5%) i/s -     49.298M in   5.015227s

Comparison:
 with 2nd arg (miss): 17672268.2 i/s
 with 2nd arg (hit) : 16896579.9 i/s - same-ish: difference falls within error
 with block (hit)   : 16513922.7 i/s - same-ish: difference falls within error
 with block (miss)  :  9836030.8 i/s - 1.80x  slower
ixti commented 5 years ago

Hash#fetch with second arg being a constant is known to be the fastest way. It was already discussed and actually has a pretty clear disclaimer/explanation:

Note that the speedup in the block version comes from avoiding repeated construction of the argument. If the argument is a constant, number symbol or something of that sort the argument version is actually slightly faster