[Create-Workload Enhancements] Rearchitect Create-Workload Feature

[IanHoang]

Overview

This is an issue based off one of the proposed priorities in this RFC: https://github.com/opensearch-project/opensearch-benchmark/issues/395

Background

As of now, OSB's create-workload is a monolith that uses a two modules of functions to create a custom workload. It was inherently designed to be a quick and easy way to build custom workloads off of small corpora. While this approach has worked in the past, there is an increasing demand for building custom workloads based off of complex workloads and more users are using this feature to achieve this.

Users who have been using this feature have mentioned that the create-workload code currently is difficult to extend, maintain, and, for newcomers to OSB, difficult to follow and interpret.

We should rearchitect the code to be more organized and scalable, which in turn will make it easier to extend and maintain. This work will also serve as the foundation for future development, such as extracting a random sampling of the documents and repairing incomplete workloads.

Proposed Design

While the existing approach is considered modular, create-workload in its current state is unwieldy. We have gathered feedback from users who have extended the feature and have used the feature to build custom workloads based on complex production workloads that are up to 10TB. Based on the feedback received, we should rearchitect create-workload to have the following components:

• CustomWorkload will be a composite class that orchestrates components such as IndexExtractor, CorpusExtractor, RetryController / RepairController, and TemplateProcessor. It can utilize helper functions to assist with processing queries provided by the user
• IndexExtractor will be a class that extracts index mappings and will not need an abstract class
• CorporaExtractor will be an abstract class that extracts data corpora. Subclasses will be one layer deep in inheritance and will include SequentialExtractor and eventually a ConcurrentExtractor. These subclasses will consist of methods that sequentially extract data and concurrently extract data. Since users have also expressed concern over Python's ability to do concurrency, we can also build in support to have a distinction between different language extractors (such as PythonCorporaExtractor or GoCorporaExtractor). This will need to be thoroughly thought out and is out of scope for now.
• RetryController / RepairController will be class that deals with incomplete workloads or index and data corpora extractions that failed and will offer users ways to retry the extraction process. This could also be more involved if users want to have a retry-create-workload failures and specify the indices that failed and this controller would grab serialized objects and retry. This is out of scope for now.
• There can be additional classes in a helper file that will serve to do miscellaneous, small tasks such as processing queries or processing workload templates.

Proposed priority

It also makes it difficult for newcomers to come and understand the code easily. This approach would promote encapsulation and abstraction, overall making create-workload more organized and scalable as well as will be easier to extend and maintain.

[IanHoang]

Received feedback to add support for pbzip2 compression now that OSB supports it. Will create a separate PR for it.

[gkamat]

@IanHoang, it may be helpful to add some child tasks to this issue, since there are multiple items here.