search

opensearch-project / opensearch-benchmark

OpenSearch Benchmark - a community driven, open source project to run performance tests for OpenSearch
https://opensearch.org/docs/latest/benchmark/
Apache License 2.0
108 stars 75 forks source link

[DISCUSSION] Update OSB Release Cadence and Method #579

Open IanHoang opened 3 months ago

IanHoang commented 3 months ago

Is your feature request related to a problem? Please describe

Right now, OSB has been releasing on a monthly cadence. This has been useful for the community as it makes OSB releases predictable and users can plan around OSB's release schedule when contributing changes.

However, the monthly release cadence only gives users a 3 week window (considering how a few days are dedicated to freeze). This is not enough time for users who are working on larger and more impactful changes in OSB.

Describe the solution you'd like

OSB maintainers would like to change two characteristics of the OSB release. 1. Release OSB every 6 weeks instead of on a monthly basis. This would align us with the OpenSearch release cadence and also allow users to have ample time to add changes in before each release. 2. Release from major / minor branches instead of from mainline. Right now, OSB release process starts once a tag is pushed up to mainline and after the release has been completed, maintainers create a major / feature branch and backport new commits when necessary. After our experience with a couple patches in 1.7.0 and 1.3.0, it would be much easier to push tags and release from major / minor branches. This would also remove the need for any freezes and would give users more time to contribute changes up until the release date.

We would like to invite the community's opinion on this matter. Feel free to comment below!

getsaurabh02 commented 3 months ago

Thanks @IanHoang. I agree with both points as they align with the OSS release cadence and branching strategy.

As an outcome of this discussion, let's publish a release calendar for OSB. This will provide better visibility for component contributors, aiding them in planning their benchmarking strategy effectively.

IanHoang commented 3 months ago

Added a forum post for visibility: https://forum.opensearch.org/t/update-opensearch-benchmark-osb-release-cadence/20126

gkamat commented 3 months ago

Both suggestions are good ones. The original reason for the monthly cadence was the spurt of incoming changes the project needed to deal with a few months ago and the need for a quick turnaround that several users wanted at the time. The rate of updates is now more steady. If we are aligned with the OpenSearch release cadence, predictability of releases and the ability for users to plan will not be impacted. The release calendar could technically be the same as the OpenSearch one, but publishing a separate one will give us the ability to diverge when necessary.

Releasing from the eponymous branch is more intuitive for users and eliminates the need for freezes as already mentioned. It will also simplify the release process.

Another relevant item would be associating release managers with corresponding releases. Initially, these individuals will be maintainers. Sharing the responsibility for putting out releases will help improve the release process and spread out release tasks more equitably. Subsequently, other community members may be able to take on release responsibilities as well.