search

akopytov / sysbench

Scriptable database and system performance benchmark
GNU General Public License v2.0
6.06k stars 1.08k forks source link

Is the test result different when I run sysbench 0.4 and 0.5 in the same enviroment? #9

Closed wangkirin closed 9 years ago

wangkirin commented 9 years ago

hi , I just started to use sysbench and i wonder if the test result will be different when I run sysbench 0.4 and 0.5 in the same enviroment (same machine and same os)?

akopytov commented 9 years ago

If we are talking about database tests, then both 0.4 and 0.5 generate exactly the same workloads with default OLTP setttings.

However, the CPU time consumed by sysbench itself may differ between 0.4 and 0.5. Since the database server is usually on the same machine where sysbench is running, one may see different results.

Generally it is not recommended to compare database results generated by different sysbench versions, even if they only differ in a minor version number (like 0.4.12 vs 0.4.13), or have been compiled from different git snapshots. As in, if you plan to perform a series of benchmarks to compare specific results, it's best to fix the sysbench binary which is used in all benchmark series.

wangkirin commented 9 years ago

thx! you mean if i run cpu or memory or file I/O test that might have different results?

akopytov commented 9 years ago

For cpu/memory/fileio there may be no differences in results, as the code is mostly the same between 0.4 and 0.5. Still, it's better to be on the safe side and use same versions to be sure you are comparing apples to apples.