opensearch-project / opensearch-benchmark

OpenSearch Benchmark - a community driven, open source project to run performance tests for OpenSearch
https://opensearch.org/docs/latest/benchmark/
Apache License 2.0
107 stars 75 forks source link

Rename test_procedures #308

Open IanHoang opened 1 year ago

IanHoang commented 1 year ago

Is your feature request related to a problem? Please describe.

Test Procedures has several forms (--test-procedure, test_procedure, and TestProcedure) throughout the CLI, workloads, and code base. It can be renamed to make it more concise.

For example, users have to run opensearch-benchmark list test_procedures to see test procedures. Underscores create confusion based on some feedback in the community. We should avoid using underscores as much as possible and streamline to use hyphens if needed. However, simplifying test procedures term would remove the need for any hyphen or underscore.

Describe the solution you'd like Could be reduced to procedure or renamed to scenarios. Open to anything

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context or screenshots about the feature request here.

beaioun commented 10 months ago

image Hi, @IanHoang I get prompted with this error saying that test-procedures is not a valid list choice, is this being deprecated or this is an issue that needs to be fixed?

IanHoang commented 10 months ago

Test procedure relates to a specific workload. It's not a part of the list feature. You'll need to use info instead of list, and specify which test-procedure you want to learn about from which workload. Here's an example:

opensearch-benchmark info --test-procedure=search-pipeline --workload=http_logs

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

Showing details for workload [http_logs]:

* Description: HTTP server log data
* Documents: 247,249,096
* Compressed Size: 1.2 GB
* Uncompressed Size: 31.1 GB

===============================
TestProcedure [search-pipeline]
===============================

Indexes the whole document corpus using OpenSearch default settings. We only adjust the number of replicas as we benchmark a single node cluster and Benchmark will only start the benchmark if the cluster turns green. Runs the search request through an search pipeline with predefined search processors.

Schedule:
----------

1. delete-index
2. create-index
3. check-cluster-health
4. create-http-log-baseline-search-pipeline
5. create-http-log-status-filter-search-pipeline
6. create-http-log-rename-field-search-pipeline
7. create-http-log-rename-100-field-search-pipeline
8. create-http-log-dummy-scripting-search-pipeline
9. create-http-log-100-dummy-scripting-search-pipeline
10. create-http-log-all-processors-search-pipeline
11. index-append (8 clients)
12. refresh-after-index
13. force-merge
14. refresh-after-force-merge
15. wait-until-merges-finish
16. match-all
17. match-all-baseline-search-pipeline
18. match-all-status-filter-search-pipeline
19. match-all-rename-field-search-pipeline
20. match-all-rename-100-field-search-pipeline
21. match-all-dummy-scripting-search-pipeline
22. match-all-100-dummy-scripting-search-pipeline
23. match-all-all-processors-search-pipeline
24. multi-term-filter
25. term-status-filter-search-pipeline
26. range
27. range-all-processors-search-pipeline

-------------------------------
[INFO] SUCCESS (took 2 seconds)
-------------------------------
maddox05 commented 10 months ago

hi. I am ready to take this issue