Gosbench is the Golang reimplementation of Cosbench. It is a distributed S3 performance benchmark tool with Prometheus exporter leveraging the official Golang AWS SDK
Gosbench consists of two parts:
INFO: -d
activates debug logging, -t
activates trace logging
go install github.com/mulbc/gosbench/server
server -c path/to/config.yaml
- you can find an example config in the example foldergo install github.com/mulbc/gosbench/worker
worker -s 192.168.1.1:2000
Make sure your prometheus configuration looks similar to this:
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets:
- localhost:9090
- job_name: 'gosbench'
scrape_interval: 1s
static_configs:
- targets:
- WORKER1.example.com:8888
- WORKER2.example.com:8888
To reload the configuration, you can either send a SIGHUP to your prometheus server or just restart it ;) Afterwards ensure that you have your Gosbench workers listed at http://your.prometheus.server.example.com:9090/targets
It is expected that the workers are in state DOWN
most of the time... they are only scrapeable during a test run.
It is Best Practice to run the Prometheus Node Exporter on all hosts as well, to gather common system metrics during the tests. This will help you in identifying bottlenecks. Please consult the Node Exporter manuals on how to install and configure it on your platform.
During a test, Prometheus will scrape the performance data continuously from the workers. You can visualize this data in Grafana. To get an overview of what the provided data looks like, check out the example scrape.
There is also an example Grafana dashboard that you can import and use. The Dashboard has some basic overview of the most common stats that people are interested in:
There are now Docker container images available for easy consumption:
docker pull quay.io/mulbc/goroom-server
docker pull quay.io/mulbc/goroom-worker
In the k8s
folder you will find example files to deploy Gosbench on Openshift and Kubernetes.
Be sure to modify the ConfigMaps in gosbench.yaml
to use your S3 endpoint credentials.
Due to popular demand, reading pre-existing files have been added. You activate this special mode by setting existing_read_weight
to something higher than 0.
There are some important things to consider though ;)
Just like with other operations, the bucket_prefix
value will be evaluated to determine the bucket name to search for pre-existing objects.
Example: This is an excerpt of your config:
objects:
size_min: 5
size_max: 100
part_size: 0
# distribution: constant, random, sequential
size_distribution: random
unit: KB
number_min: 10
number_max: 100
# distribution: constant, random, sequential
number_distribution: constant
buckets:
number_min: 2
number_max: 10
# distribution: constant, random, sequential
number_distribution: constant
bucket_prefix: myBucket-
Note: Due to the constant distribution, we will only consider the _min
values.
This will cause each workers to search for pre-existing files in the buckets myBucket-0
and myBucket-1
and read 10 objects from these buckets. If there are less than 10 objects in any of these buckets, some objects will be read multiple times. The object size given in your config will be ignored when reading pre-existing files.
When a new tool is presented, it’s essential to compare it to existing tools for accuracy. For this reason, we ran a comparision between Cosbench and Gosbench. Both benchmarks were tasked to do a 100% write test and 100% read test on 4KB, 16KB, 256KB, 1MB, 4MB objects for 60 seconds each. The tests were to run on one RGW using S3 protocol in ceph storage clusteri, also run in the test configuration in parallel. Figure below show writing and reading, respectively. From these charts, it’s apparent that the performance metrics for all objects are similar for both tools.
pre-commit install