elastic / rally

Macrobenchmarking framework for Elasticsearch
Apache License 2.0
37 stars 314 forks source link

Refactor version number checks #1738

Closed b-deam closed 1 year ago

b-deam commented 1 year ago

Serverless Elasticsearch doesn't return a version.number field from the Info API (/) , which is actually something we rely on somewhat commonly throughout the codebase.

To fix this, we need a clearly defined way of determining whether or not Rally is talking to a Serverless Elasticsearch, this applies for both clients used to target the system under test, as well as any ancillary clients like those used by a remote metrics store.

In order to do so, we'll take two approaches:

I think this is a balanced approach, given that serverless has no concept of 'versioning'.

Note:

The underlying client's use of .options() complicates things, because every call reinstantiates a new copy of itself, but without the ability for us to pass in any custom arguments (e.g. distribution_version). To work around this we must work out upfront whether or not we'd like the factory to create a serverless or non-serverless client, each of which have a static is_serverless attribute. Fixed in https://github.com/elastic/rally/pull/1738/commits/abf43207e1ebde6899876dcc3ebadf5998e7eab1

Example tests ``` # http_logs with node-stats and data-stream-stats telemetry devices esrally race --track http_logs --pipeline=benchmark-only --target-hosts https://serverless-es --client-options='{ "default": {"headers": {"X-Found-Cluster": "09895e10-efb8-42da-be30-888eb623d6cf.es"}, "basic_auth_user":"esbench", "basic_auth_password":"changeme", "verify_certs": false}}' --kill-running-processes --test-mode --track-params 'number_of_replicas: 1' --telemetry "node-stats, data-stream-stats" # create-track esrally create-track --track test --indices logs-221998 --target-hosts https://serverless-es --client-options '{ "default": {"headers": {"X-Found-Cluster": "09895e10-efb8-42da-be30-888eb623d6cf.es"}, "basic_auth_user":"elastic", "basic_auth_password":"changeme", "verify_certs": false}}' ```
pquentin commented 1 year ago

buildkite build this please after https://github.com/elastic/rally/pull/1739