evanphx / benchmark-ips

Provides iteration per second benchmarking for Ruby
MIT License
1.72k stars 97 forks source link

Improve report to show duration/i (e.g: `12.34 ns/i`). #132

Closed huacnlee closed 9 months ago

huacnlee commented 9 months ago

After update, the new report will look like:

ruby 3.2.0 (2022-12-25 revision a528908271) +YJIT [arm64-darwin23]
Warming up --------------------------------------
          string.new     1.485M i/100ms
       string.concat     1.086M i/100ms
        string.split   649.289k i/100ms
Calculating -------------------------------------
          string.new     15.947M (± 1.2%) i/s   (62.71 ns/i) -     32.665M in   2.048640s
       string.concat     10.846M (± 0.7%) i/s   (92.20 ns/i) -     21.718M in   2.002441s
        string.split      6.506M (± 0.3%) i/s  (153.70 ns/i) -     13.635M in   2.095740s

In some time we need to know how fast (in avg time), the before report only show the iterations/s, but we don't know how long of each iteration.

So I add the duration of each iteration in the report.

This may be useful to compare the performance with other languages, (e.g. Go, Rust etc.) they have the duration in per iteration.

For example Go:

Benchmark_string_alloc-8                1000000000           0.3361 ns/op

Use this code to test result:

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"
  gem "benchmark-ips", github: "huacnlee/benchmark-ips", branch: "feat/report-duration-per-iter"
end

require "benchmark/ips"

text = "hello world"
part = "world"

Benchmark.ips do |x|
  x.config(time: 2, warmup: 1)

  x.report("string.new") do
    a = "hello world"
  end
  x.report("string.concat") do
    text + part
  end
  x.report("string.split") do
    text.split(" ")
  end
end
nateberkopec commented 9 months ago

I like the feature, but I'm not sure about the presentation of it.

I think I would prefer 22.780M (± 4.9%) i/s (43.90 ns/i) - 46.044M in 2.027883s)

huacnlee commented 9 months ago

I have updated, and I left 15 width for (duration/i) for tidy print.

The new result:

Warming up --------------------------------------
          string.new     1.485M i/100ms
       string.concat     1.086M i/100ms
        string.split   649.289k i/100ms
Calculating -------------------------------------
          string.new     15.947M (± 1.2%) i/s   (62.71 ns/i) -     32.665M in   2.048640s
       string.concat     10.846M (± 0.7%) i/s   (92.20 ns/i) -     21.718M in   2.002441s
        string.split      6.506M (± 0.3%) i/s  (153.70 ns/i) -     13.635M in   2.095740s