[DOC] Versioned Data Prepper documentation

[dlvenable]

What do you want to do?

• [x] Request a change to existing documentation
• [ ] Add new documentation
• [ ] Report a technical problem with the documentation
• [ ] Other

Tell us about your request. Provide a summary of the request and all versions that are affected.

Data Prepper is a distinct product from OpenSearch. It has its own versions and release schedule. And similar to OpenSearch, there are incompatibilities between major versions.

The OpenSearch documentation currently allows readers to select an OpenSearch version. However, this is not possible with Data Prepper. So readers are unable to know what was valid in Data Prepper 1.5 for example. They only know the latest.

The solution here should be flexible to support any other artifacts with different release cycles. It could be that we want to take a similar approach with the Logstash plugins (maybe, maybe not).

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

N/A

[Naarcha-AWS]

Target Date: Mid-November

[dlvenable]

This proposal would likely change the overall structure of the documentation page.

Right now, Data Prepper documentation is located at the following: https://opensearch.org/docs/latest/clients/data-prepper. Readers change change the OpenSearch version to other versions, say https://opensearch.org/docs/2.2/clients/data-prepper.

If the documentation resided at the same location, it would result in double versions in the URL and create an explosion of possible paths. For example, OpenSearch 2.2 and Data Prepper 2.0: https://opensearch.org/docs/2.2/clients/data-prepper/2.0/. This would not be a good experience.

Thus, I believe that the Data Prepper documentation should be moved to a top-level page. There are a couple possible options here.

Option 1 - Move Data Prepper only

Move the Data Prepper documentation such that it gets its own page.

https://opensearch.org/docs/data-prepper/latest
https://opensearch.org/docs/data-prepper/2.0

or

https://opensearch.org/data-prepper/docs/latest
https://opensearch.org/data-prepper/docs/2.0

Option 2 - Move OpenSearch documentation as well

The OpenSearch documentation could be given a sub-path as well. This way both products have similar path structures.

https://opensearch.org/docs/data-prepper/latest
https://opensearch.org/docs/data-prepper/2.0
https://opensearch.org/docs/opensearch/latest
https://opensearch.org/docs/opensearch/2.2

Overall, I'd like to suggest that the documentation take the approach of Option 1 since this would not break any existing URLs for OpenSearch. The downside is it may add some development work because the docs page will need to detect if the path is another product (e.g. data-prepper) or an OpenSearch version (e.g. 2.2) in the base URL https://opensearch.org/docs/{subProductOrOpenSearchVersion}.

[AMoo-Miki]

@kgcreative, would love to hear your thoughts; how would we interface these docs on the website?

[kgcreative]

@AMoo-Miki Lets generalize this to be able to accommodate Data Prepper, OpenSearch and OpenSearch Dashboards

[Naarcha-AWS]

We can close this issue after the following items are complete:

• [x] New Data Prepper repo is created.
• [ ] All issues labeled data-prepper are transferred to the new repo.
• [ ] Come up with a solution for accounting of changes in plugins between versions.
• [ ] Add a breaking changes page to the repo (Add this as a separate issue for 3.0.0)

[dlvenable]

We are going to support versioned Data Prepper documentation with a new documentation repo specifically for Data Prepper documentation. This will decouple the branches and versioning in that repository from those of this project (documentation-website).

[hdhalter]

Hi @dlvenable David, since we decoupled the Data Prepper content from the OS versioning, do you feel comfortable closing this issue? https://opensearch.org/docs/latest/data-prepper/. We eventually want to create a separate pipeline (and potentially a separate repo) and version the content, but can open new issues for those things, as needed.

[dlvenable]

@hdhalter , This issue also requests that we have versioning for the Data Prepper documentation. The decoupling helps towards this. But, I think we will still need multiple Data Prepper documentation versions. When Data Prepper 3.0 comes out, it would be good to have the 2.x documentation still available. So I think we should keep this issue open while we work toward that.

[hdhalter]

@hdhalter , This issue also requests that we have versioning for the Data Prepper documentation. The decoupling helps towards this. But, I think we will still need multiple Data Prepper documentation versions. When Data Prepper 3.0 comes out, it would be good to have the 2.x documentation still available. So I think we should keep this issue open while we work toward that.

Sounds good!