elastic / rally

Macrobenchmarking framework for Elasticsearch
Apache License 2.0
1.95k stars 313 forks source link

Fix serverless public use in trivial tracks #1785

Closed gbanasiak closed 1 year ago

gbanasiak commented 1 year ago

Eliminates the following blocking points for Rally public use against serverless clusters:

With the fixes in it's now possible to run trivial benchmarks with delete-index, create-index and bulk operations.

% esrally race --track-path=~/tracks/test_track --target-hosts=${ES_HOST}:443 --pipeline=benchmark-only --client-options="use_ssl:true,api_key:${ES_API_KEY}" --on-error=abort

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Race id is [bc7d2dcc-b176-4a79-a293-8f53b97e9fd2]
[INFO] Detected Elasticsearch Serverless mode with operator=[False].
[INFO] Excluding [cluster-health] as challenge [default] is run on serverless.
[INFO] Racing on track [test_track] and car ['external'] with version [oss].

Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running bulk                                                                   [100% done]

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------

|                        Metric |   Task |   Value |   Unit |
|------------------------------:|-------:|--------:|-------:|
|                Min Throughput |   bulk | 1345.31 | docs/s |
|               Mean Throughput |   bulk | 1345.31 | docs/s |
|             Median Throughput |   bulk | 1345.31 | docs/s |
|                Max Throughput |   bulk | 1345.31 | docs/s |
|       50th percentile latency |   bulk | 1929.95 |     ms |
|      100th percentile latency |   bulk | 3366.24 |     ms |
|  50th percentile service time |   bulk | 1929.95 |     ms |
| 100th percentile service time |   bulk | 3366.24 |     ms |
|                    error rate |   bulk |    0    |      % |

--------------------------------
[INFO] SUCCESS (took 44 seconds)
--------------------------------

There are many index settings that will still prevent completion of benchmarks from https://github.com/elastic/rally-tracks but this will be addressed separately.

pquentin commented 1 year ago

I'm not sure why this is a draft, this is a good step forward, as shown by the serverless IT tests that report a different error now.

gbanasiak commented 1 year ago

@pquentin Many thanks for a quick review. I've left it in draft mode while waiting for the tests to complete, but I wasn't planning to add anything.