Reduce code duplication on 'target_throughput' and 'search_clients' parameters definition

tlfeng commented 1 year ago

Is your feature request related to a problem?

Code duplication is introduced by the PR https://github.com/opensearch-project/opensearch-benchmark-workloads/pull/47

The below is the redundant code, which exists in the test procedure definition file for all datasets.

{%- if not target_throughput %}
,"target-throughput": 3
{%- elif target_throughput is string and target_throughput.lower() == 'none' %}
{%- else %}
,"target-throughput": {{ target_throughput | tojson }}
{%- endif %}
{%-if search_clients is defined and search_clients %}
,"clients": {{ search_clients | tojson}}
{%- endif %}

What solution would you like?

By using macros syntax, like commit https://github.com/opensearch-project/opensearch-benchmark-workloads/pull/58/commits/2a7138d050b645a0204334ad9f0a252d2b494889, to reuse the common codes in every file.

What alternatives have you considered?

Put the common codes in a separate file for reusing.

Do you have any additional context?

Coming from https://github.com/opensearch-project/opensearch-benchmark-workloads/pull/58#discussion_r1125144180

gaiksaya commented 1 year ago

[Triage] @gkamat Can you looks into this? Thanks!

gkamat commented 1 year ago

We will address this shortly.

tlfeng commented 1 year ago

@gkamat @IanHoang I had a proposed code change to resolve the issue in PR https://github.com/opensearch-project/opensearch-benchmark-workloads/pull/82, but I closed it. Noticed you are refactoring the benchmark workload, I don't want to disturb your plan by adding massive code change. I can open the PR again when you want to deal with this problem, or that's fine if you want to make code changes for this issue by yourself. 😁

IanHoang commented 12 months ago

@gkamat has made some recent changes to percolator and PMC workloads. For future workloads, we can follow the convention he applied:

opensearch-project / opensearch-benchmark-workloads