Closed patsonluk closed 11 months ago
@chatman Adding ability to export benchmarking metrics via grafana /metrics endpoint (different from the existing prometheusMetrics
collector which collects prometheus metrics from the current host)
Also tagging @nginthfs and @hiteshk25 😊
Fixed some inconsistent whitespace vs tab and indentation in the files, please use https://github.com/fullstorydev/solr-bench/pull/84/files?w=1 for code review!
Changes look good! Thanks..
Is it possible to include a sample configuration exposing some of these changes? Not a blocker, we can do it later also.
Thank you so much for the review!
As for the config, I thought of adding new one or changing existing one, but it could be a bit hard for users as it's not super trivial which config would have the settings.
As a part of this PR, we did add description to the README.md and the sample usage: https://github.com/fullstorydev/solr-bench/pull/84/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R88 .
Description
Currently solr-bench export test results to a json file upon test completion. However, it would also be useful to have the test metrics be exposed via Prometheus /metrics endpoint such that data can be graphed by other tools such as Grafana
Solution
Added new config parameter
prometheus-export
to the workflow (top-level) config.port
is the port used for prometheus /metrics endpoint (default to 11100, take note that we picked this to avoid the default 9100 port used for prometheus node-exporter)type-label
is a custom label that user can define to better group metrics. default tounknown
. There's also an overrideprometheus-type-label
underquery-benchmark
in case if user wants more granular control on such label to per benchmark setup.To enable Prometheus with defaults, simply add
"prometheus-export": {}
Histogram
is used to store the metrics with labels:method
: the http Method used for the callpath
: the relative path used . For example/select
,/update
etctype
: the custom label described above. Default tounknown
The current implementation explicitly declares the metrics to be captured (as histogram) and only capture metrics from
QueryBenchmark
andIndexBenchmark
. Other benchmarking tasks are not prometheus enabled ATM.Take note that the Prometheus endpoint server would only be started if
prometheus-export
is defined with valid metrics to export. (ie if the benchmark only does restart, it wouldn't start the prometheus export endpoint)Remarks
This PR also adds some extra features, fixed some bugs:
stress.sh
would now read the json config file with a new paramextra-jvm-args
, which the value will be added to the java startup call. This is useful for SSL support