root-project / rootbench

Collection of benchmarks and performance monitoring applications
GNU Lesser General Public License v2.1
20 stars 41 forks source link

ROOT Benchmarks

This repository contains a set of relatively small programs (usually based on gbenchmark micro benchmarking infrastructure) built on top of ROOT. Their primary goal is to provide stable performance metrics which can be monitored over time.

Results of nightly run of ROOT benchmarks could be discovered in the Rootbench.git Grafana instance: RootBench Grafana.

Project Health

Linux/OSX Build Status Experimental Benchmark Coverage: Coverage Status

Cite

@inproceedings{shadura2019continuous,
  title={Continuous Performance Benchmarking Framework for ROOT},
  author={Shadura, Oksana and Vassilev, Vassil and Bockelman, Brian Paul},
  booktitle={EPJ Web of Conferences},
  volume={214},
  pages={05003},
  year={2019},
  organization={EDP Sciences}
}

Building

ROOTBench can be built standalone and as part of ROOT. If you want to enable ROOTBench for ROOT just add the -Drootbench=On option to your cmake configuration.

Building ROOTBench standalone

ROOTBench should be able to find ROOT at configuration time. Make sure you ran source $ROOTSYS/bin/thisroot.sh.

git clone https://github.com/root-project/rootbench.git
mkdir build
cd build
cmake ../rootbench
cmake --build . -- -j$(nproc)

also, you can use

cmake --build . -- -jN

where 'N' is the maximum number of processor cores you want to use.

Extending the benchmarks

ROOTBench relies on Google Benchmark. We recommend to read the available documentation and browse the existing examples here for more advanced usage.

Background

This repository is being integrated in two steps:

The integration process depends on the overall benchmarking time. Contributors are encouraged to write well-focused microbenchmarks ensuring good benchmarking coverage. Non-overlapping microbenchmarks seem to be the only reasonable way to control the pressure on the infrastructure.

Conventions

There are several practical conventions that we should follow:

Simple benchmark template

Add file called CLASSNAMEBenchmarks.cxx where CLASSNAME is the name of the ROOT class we benchmark.

#include "ROOT_HEADER_TO_BENCHMARK.h"

#include "benchmark/benchmark.h"

// Replace the CLASSNAME and ROUTINE with the ROOT class and routine you are benchmarking respectively. 
static void BM_CLASSNAME_ROUTINE(benchmark::State &state) {
  // Initialization section before actual benchmarking.
  for (auto _ : state) {
    // The benchmarking code goes here.
  }
  // Teardown.
}
BENCHMARK(BM_CLASSNAME_ROUTINE);

// In the end of the file we add our main().
BENCHMARK_MAIN();

Register the benchmark in the system. Add an entry to the CMakeLists.txt next to the source code of the benchmark.

RB_ADD_GBENCHMARK(CLASSNAMEBenchmarks
  CLASSNAMEBenchmarks.cxx
  LABEL short
  LIBRARIES LIST OF LIB DEPENDENCIES)

This is a very basic working example. If you need extra functionality please read the Google Benchmark Docs.