OSB is not compatible with Shenandoah GC | Java 17

[ayushav12]

Describe the bug I created an OpenSearch-2.3 domain with updated JVM settings to use single-generation Shenandoah Garbage Collector. Then, to test my domain for performance, I setup OSB and ran nyc_taxis workload. The test run failed giving error :

in jvm_stats old_gen_collection_time = gc["old"]["collection_time_in_millis"]

KeyError: 'old'

Since, Shenandoah is single-generation GC, there's no concept on old-generation, new-generation and permanent-generation as in generic garbage collector algorithms.

More Context (please complete the following information):

• Workload(Share link for custom workloads) : nyc_taxis
• Service(E.g OpenSearch) : OpenSearch
• Version (E.g. 1.0) : 2.3

[IanHoang]

After taking a quick glance in the codebase, this is correct, OSB so far only supports multi-generational garbage collectors (like G1GC and CMS) and does not support experimental types of Garbage Collectors like Shenandoah. We'd have to look at incorporating support for single-generation GCs.

Although it's unlikely that you won't be needing it, if you still want to run node-stats on OS 2.3 without GC metrics, this can be done by running with additional parameters --telemetry-params=node-stats-include-gc=false in the OSB command.

[IanHoang]

Update: It seems like passing in false for node-stats-include-gc doesn't tell OSB to not collect GC data. It only tells OSB to not include GC data into the final node-metrics document that is produced and ingested into the metrics data store. This is why it still fails and encounters the same issue about unable to find old GC even when parameter is passed in. https://github.com/opensearch-project/opensearch-benchmark/blob/a1f45502b5b9e69bad1c8d10e7a6c30bd0ed8469/osbenchmark/telemetry.py#L706-L721

[IanHoang]

This issue is similar to #206 and should be regarded as an enhancement rather than a bug because OSB currently supports GCs with concepts of old / young generations like G1GC and CMS GCs. We've currently implemented a short-term fix but will work on a long-term fix eventually.