How to benchmark sequentially?

[ddnexus]

I need to benchmark and compare 2 blocks of code with and without the inclusion of a module. If I do something like:

def my_method
  Benchmark.ips do |x|

    x.report('without') do
      something
    end

    code_to_include_module

    x.report('with') do
      something
    end

    x.compare!

  end
end

The inclusion happens immediately, so both blocks run with the inclusion. Using hold! doesn't change it. How could I do it?

[danmayer]

I was struggling to figure out hold! as well but realized you can just call it twice and it will only measure the first block on first run and the second on second run...

def my_method
  Benchmark.ips do |x|

    x.report('without') do
      something
    end

    if ENV['WITH_MODULE']=='true'
      code_to_include_module
    end

    x.report('with') do
      something
    end
    x.hold! 'temp_results'
    x.compare!
  end
end

then run the first time like so ruby yourfile.rb which outputs

Warming up --------------------------------------
           without     1.000  i/100ms
Calculating -------------------------------------
           without      3.650  (± 0.0%) i/s -     19.000  in   5.248510s

Pausing here -- run Ruby again to measure the next benchmark...

and on the second run like so WITH_MODULE=true ruby yourfile.rb which outputs

Warming up --------------------------------------
        with     1.000  i/100ms
Calculating -------------------------------------
        with   14.200  (±14.1%) i/s -     70.000  in   5.024695s

Comparison:
        without:       14.2 i/s
             with:        3.6 i/s - 3.89x  slower

hope that helps @ddnexus

[ddnexus]

Thank you! It helps a lot!

[danmayer]

ok added full example as a PR @ddnexus https://github.com/evanphx/benchmark-ips/pull/86

[kbrock]

can this issue be closed? The example is great and seems to answer this question

[danmayer]

yes it was all merged in, so I would +1 closing this issue, I think @evanphx would need to do that ;)

[ddnexus]

Sorry for the late answer. Yes I can close the issue. THank you!