opensearch-project / opensearch-build

🧰 OpenSearch / OpenSearch-Dashboards Build Systems
Apache License 2.0
136 stars 271 forks source link

Integrate feature of increasing the data size with benchmark test script #3701

Open tlfeng opened 1 year ago

tlfeng commented 1 year ago

Is your feature request related to a problem? Please describe

The benchmark test script can be used as a convenient automated script to run performance test locally. I'd like to have it integrate the existing feature of opensearch-benchmark to expand the workload data size.

Document for the feature: https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/2/http_logs#beta-feature-increasing-the-size-of-the-data-corpus

Describe the solution you'd like

May be achieved by modifying the code around this line, which executing the openserch-benchmark command.

Here are more documents: https://github.com/opensearch-project/opensearch-benchmark/blob/1.0.0/docker/README.md?plain=1#L20 https://github.com/opensearch-project/opensearch-benchmark/blob/1.0.0/scripts/expand-data-corpus.py#L22

Here is the code I wrote to make it possible https://github.com/tlfeng/opensearch-build/commit/9df12c303851051c560ffe5c99f5ba9146bb53a7, and it can be a reference for anyone has the same demand.

Describe alternatives you've considered

No response

Additional context

No response

jordarlu commented 1 year ago

Thanks for sending the issue, @tlfeng . @rishabh6788 , pls have a look and provide your feedback.. thanks ,