search

elastic / rally

Macrobenchmarking framework for Elasticsearch
Apache License 2.0
1.95k stars 313 forks source link

Exclude telemetry devices based on serverless status #1770

Closed pquentin closed 1 year ago

pquentin commented 1 year ago

Can be tested using:

esrally race --track-path=$HOME/src/rally-tracks/geonames --test-mode --target-hosts=... --pipeline=benchmark-only --client-options=serverless.json --on-error=abort --telemetry=ccr-stats

This will run the race and print:

[INFO] Excluding telemetry device [ccr-stats] as it is unavailable on serverless.

To test without operator status, you should either include https://github.com/elastic/rally/pull/1768 or set serverless.operator to false in the rally.ini [driver] section.

TODO:

pquentin commented 1 year ago

To be clear, here is the current output:

[INFO] Race id is [de8d87c7-26d2-4caa-a8b6-5747281babb1]                                                                                                       
[INFO] Treating parallel task in challenge [indexing-querying] as public.                                                                                      
[INFO] Excluding [put-settings], [check-cluster-health], [force-merge], [wait-until-merges-finish] as challenge [indexing-querying] is run on serverless.      
[INFO] Racing on track [pmc], challenge [indexing-querying] and car ['external'] with version [8.10.0].                                                        

[INFO] Excluding telemetry device [ccr-stats] as it is unavailable on serverless.       
Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running initial-index-append                                                   [100% done]
Running refresh-after-index                                                    [100% done]
Running refresh-after-force-merge                                              [100% done]
Running phrase,articles_monthly_agg_uncached,default,term,articles_mont...     [100% done]
[INFO] Excluding telemetry device [ccr-stats] as it is unavailable on serverless.

The empty line between "Racing" and "Excluding" is annoying, but I want to remove the second "Excluding" message between the "Running" messages and the summary report.

pquentin commented 1 year ago

OK, it should be correct now! A nice win for test-driven development. Please take another look.