martinus / nanobench

Simple, fast, accurate single-header microbenchmarking functionality for C++11/14/17/20
https://nanobench.ankerl.com
MIT License
1.43k stars 82 forks source link

Feat/multi output #112

Open SGSSGene opened 4 months ago

SGSSGene commented 4 months ago

This allows to use multiple 'Bench' classes to group benchmarks together. Only print a new header if the same output stream runs a new test.

This changes the output:

|               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | at()      
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:-----                                                                                                                                                                              
|            1,280.29 |          781,073.04 |   12.2% |        2,285.81 |        5,338.52 |  0.428 |         424.05 |    2.5% |      0.01 | :wavy_dash: `std::map` (Unstable with ~647.3 iters. Increase `minEpochIterations` to e.g. 6473)

|               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | operator[]
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:-----------
|            1,333.55 |          749,878.68 |    8.0% |        2,285.69 |        5,357.32 |  0.427 |         423.92 |    2.5% |      0.01 | :wavy_dash: `std::map` (Unstable with ~765.7 iters. Increase `minEpochIterations` to e.g. 7657)

|               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | at()
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:-----
|              208.09 |        4,805,636.97 |    1.9% |          536.00 |          919.61 |  0.583 |          89.00 |    0.0% |      0.01 | `std::unordered_map`

|               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | operator[]
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:-----------
|              250.45 |        3,992,816.28 |    5.9% |          466.00 |        1,103.79 |  0.422 |          76.00 |    0.0% |      0.01 | :wavy_dash: `std::unordered_map` (Unstable with ~4,042.5 iters. Increase `minEpochIterations` to e.g. 40425)

to

|               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | at()
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:-----
|            1,262.44 |          792,117.10 |   15.0% |        2,284.03 |        4,821.46 |  0.474 |         423.68 |    2.5% |      0.01 | :wavy_dash: `std::map` (Unstable with ~656.7 iters. Increase `minEpochIterations` to e.g. 6567)
|              210.86 |        4,742,583.39 |    1.1% |          536.00 |          925.75 |  0.579 |          89.00 |    0.0% |      0.01 | `std::unordered_map`

|               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | operator[]
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:-----------
|            1,036.34 |          964,935.25 |    0.5% |        2,285.09 |        4,640.47 |  0.492 |         423.85 |    2.5% |      0.01 | `std::map`
|              197.68 |        5,058,664.21 |    1.0% |          466.00 |          878.61 |  0.530 |          76.00 |    0.0% |      0.01 | `std::unordered_map`

More details on the intended use can be seen in src/test/unit_multi_bench.cpp