Benchmarks were previously run in an ad hoc manner with GTest with inconsistent output. Using a formal benchmark library standardizes the output, making it easier to parse and analyze, and automatically adds timing measurements.
Implementation
The library itself is fairly similar to GTest. Two approaches are used here for benchmarks: a fixture-based approach and a standard function-based approach. The state for each benchmark consists of an integer value and it can be used for configuration. The benchmark itself only starts timing after entering the for (auto _ : st) { ... } loop.
Notes
The container will need to be rebuilt to include Benchmark. Alternatively, follow the Dockerfile instructions to install it in your existing container.
One missing feature from GTest is executable scoped fixtures (all fixtures Setup and Teardown methods run before/after each test). This creates a problem for running multiple HTTP client tests because that cannot be restarted in the same executable. The workaround is that a wrapper script will be needed to filter out the HTTP tests and run them one at a time after.
gbuildx
commented
1 year ago
Summary of Changes
Closes #146
Motivation
Benchmarks were previously run in an ad hoc manner with GTest with inconsistent output. Using a formal benchmark library standardizes the output, making it easier to parse and analyze, and automatically adds timing measurements.
Implementation
The library itself is fairly similar to GTest. Two approaches are used here for benchmarks: a fixture-based approach and a standard function-based approach. The state for each benchmark consists of an integer value and it can be used for configuration. The benchmark itself only starts timing after entering the
for (auto _ : st) { ... }loop.Notes
The container will need to be rebuilt to include Benchmark. Alternatively, follow the Dockerfile instructions to install it in your existing container.
One missing feature from GTest is executable scoped fixtures (all fixtures Setup and Teardown methods run before/after each test). This creates a problem for running multiple HTTP client tests because that cannot be restarted in the same executable. The workaround is that a wrapper script will be needed to filter out the HTTP tests and run them one at a time after.